Confirmed users
574
edits
DXR Parallel Tree Indexing: Difference between revisions
Add compound ID.
(Add generated_date.) |
(Add compound ID.) |
||
| Line 16: | Line 16: | ||
But how do we know which of the trees in the config file are actually indexed so far? IOW, what if someone adds a new tree, and it takes awhile to index? Or what if somebody enables a new plugin for that tree, and we don't want to start showing filters for it until it's actually been used in an indexing run? We need to freeze certain attributes of a tree as they were at index time. We'll keep the list of these "frozen" attributes in a dedicated index that has 1 shard(?) and replicas all over the place so queries are fast. The docs would look like this: | But how do we know which of the trees in the config file are actually indexed so far? IOW, what if someone adds a new tree, and it takes awhile to index? Or what if somebody enables a new plugin for that tree, and we don't want to start showing filters for it until it's actually been used in an indexing run? We need to freeze certain attributes of a tree as they were at index time. We'll keep the list of these "frozen" attributes in a dedicated index that has 1 shard(?) and replicas all over the place so queries are fast. The docs would look like this: | ||
{name: "mozilla-central", | {id: '11/mozilla-central', # "compound index" of format and name, to rule out any possibility of duplication | ||
name: "mozilla-central", | |||
format: 11, # By storing the number here, we can just query for.... We don't have to worry about the alias template string going stale; the deploy script re-reads the config file each time. | format: 11, # By storing the number here, we can just query for.... We don't have to worry about the alias template string going stale; the deploy script re-reads the config file each time. | ||
es_alias: "dxr_11_mozilla-central", # in case es_alias changes in the conf file | es_alias: "dxr_11_mozilla-central", # in case es_alias changes in the conf file | ||