Metadata Lookups
Overview
Lookup tables are useful to reduce the size of your overall data set and to make sure that id/name combinations are always available with the latest data. Lookup examples would include:
- Campaign ID -> Campaign Name
- Country Code -> Country
- Account Owner -> Owner Name
The best implementation path for lookups is to place the an output file (typically jsonl or csv) in an s3 or gcs bucket and point Druid towards that location.
Adding Lookups
Within Druid, you can access Lookups from the main control page (either input the URL for our console or select the Druid logo on the top left to return to the homepage. From that page, select Lookups*
On the next page, select Add Lookups* to bring up the entry screen
On the Lookup entry screen, the first key change is the type - changing from map to cachedNamespace
Changing the Type setting will bring up the necessary options to configure the lookup. From here enter:
URL string of the location in the URI prefix Change the parse format to the filetype *Update the polling period depending on frequency of change
Lookup files can contain a single id to name mapping or multiple fields. When selecting the parse format (either JSON or CSV) you have the option to define the Key and Value fields that are returned.
If your lookup is 1:1 meaning each ID has only one value, changing the setting to Injective = True will improve performance during query time