ConceptSetManifest R6 Class
ConceptSetManifest R6 Class
Details
An R6 class that manages a collection of ConceptSetDef objects and maintains metadata in a SQLite database.
The ConceptSetManifest class manages multiple concept set definitions and stores their metadata in a SQLite database located at inputs/conceptSets/conceptSetManifest.sqlite. Each ConceptSetDef is assigned a sequential ID based on its position in the manifest.
By default, the load file is treated as a transient, one-time import
mechanism: rows whose atlasId is already registered in the manifest are
an error, not an update. Set stopIfExists = FALSE to instead update
those rows in place, which supports iterating on the load file across
repeated runs. To sync registered concept sets with ATLAS without a
load file, use updateAtlasConceptSets().
Processing Steps:
Validates that all concept set IDs exist and are active
Loads each concept set JSON as a Capr object using
Capr::readConceptSet()Combines them using set logic (private helper
combine_capr_concept_sets())Exports combined Capr object to JSON in
json/directoryRegisters the new combined concept set in the manifest
Returns the new concept set ID
Concept Set Combination Logic:
Includes: All included concepts across sets
Descendants: All concepts marked with descendants
Excludes: All excluded concepts (without descendants)
Exclude+Descendants: All concepts to exclude with descendants
Requirements:
Capr package must be installed
All source concept sets must be active and have valid JSON files
This is the detection phase of the ATLAS maintenance workflow. Use this to identify
which ATLAS concept sets have changed, then optionally call updateAtlasConceptSets() to
apply updates. Changes are detected by comparing expression JSON hashes.
This method updates ATLAS concept sets that have changed in the remote repository. It:
Calls checkAtlasConceptSets() to identify changes
For each changed concept set: fetches current definition, updates JSON file, updates hash in manifest
Refreshes the in-memory manifest
Use checkAtlasConceptSets() first to identify which concept sets have changed, then call
this method to apply updates.
Requirements:
ExecutionSettings must be initialized with a valid database connection
ExecutionSettings must have
cdmDatabaseSchemaand optionallytempEmulationSchemasetUser must have READ access to OMOP concept and concept_ancestor tables
Processing:
Resolves the concept set by manifest ID or exact label
Retrieves the concept set definition (CIRCE JSON)
Builds SQL query using
CirceR::buildConceptSetQuery()Executes query against the OMOP vocabulary schema
Returns results with concept_id and concept_name columns
Extract Source Codes for Concept Sets
Vocabulary Suggestion by Domain: The function automatically suggests appropriate vocabularies based on concept set domains:
condition_occurrence: ICD10CM, ICD9CMprocedure: HCPCS, CPT4measurement: LOINCdrug_exposure: NDCobservation: All vocabularies (ICD9CM, ICD10CM, HCPCS, CPT4, LOINC, NDC)device_exposure: NDCvisit_occurrence: ICD10CM, ICD9CM, HCPCS, CPT4
Note: These suggestions are based on OMOP CDM conventions. You can override with any valid vocabulary combination.
Processing Workflow:
Verifies ExecutionSettings is configured with database connection
Detects domains of all concept sets in the manifest
Displays suggested vocabularies based on detected domains
Prompts user to accept or override suggested vocabularies
Creates a new xlsx workbook
For each concept set in the manifest:
Reads the CIRCE JSON definition
Builds a concept query selecting standard concepts (using CirceR)
Performs SQL join: concepts -> concept_relationship (Maps to) -> source concepts
Finds matching source codes in the specified vocabularies
Adds results as a new sheet in the xlsx workbook with formatted header
Provides status messages for each concept set
Exports combined results to
{outputFolder}/SourceCodeWorkbook.xlsxEach sheet contains columns: vocabulary_id, concept_code, concept_name
Sheet headers are styled with blue background and white bold text
Column widths are auto-fitted for readability
SQL Query Pattern: For each concept set, the following logic is executed:
CTE selects all standard concepts in the concept set
Joins to concept_relationship table with relationship_id = 'Maps to'
Maps relationship finds what source codes map TO standard concepts
Filters to valid, non-invalid source codes in specified vocabularies
Results ordered by vocabulary_id and concept_code
Requirements:
ExecutionSettings must be initialized with a valid database connection
Vocabulary schema must be accessible from ExecutionSettings
openxlsx2 package must be installed
User must have READ permissions on vocabulary tables
Error Handling:
Displays warnings if any concept set processing fails but continues with others
Provides clear error messages if database connection is unavailable
Validates source vocabularies against known vocabulary IDs
This function identifies which standard concepts are included in each concept set by finding the reverse mapping relationship. For each concept set:
Reads the CIRCE JSON definition
Builds a concept query using CirceR
Joins with concept_relationship via reverse "Maps to" relationship (finds what maps TO the concept set concepts)
Filters for standard concepts (standard_concept = 'S')
Adds results to a new sheet in the xlsx workbook
Exports all results to
{outputFolder}/IncludedCodes.xlsxEach sheet contains: concept_id, concept_name, vocabulary_id
Requirements:
ExecutionSettings must be initialized with a valid connection
Vocabulary schema must be accessible from ExecutionSettings
openxlsx2 package must be installed
Methods
Method new()
Initialize a new ConceptSetManifest
Usage
ConceptSetManifest$new(
dbPath = "inputs/conceptSets/conceptSetManifest.sqlite",
projectRoot = NULL
)Arguments
dbPathCharacter. Path to the SQLite database. Defaults to "inputs/conceptSets/conceptSetManifest.sqlite". The directory is created automatically if it does not exist.
projectRootCharacter or NULL. Study repository root against which stored file paths are resolved. When
NULL(default) the root is discovered once viafindStudyProjectRoot()from the manifest's directory and cached for the life of the object. Supply an explicit path for tests or unusual layouts. Get the manifest as a list of ConceptSetDef objects
Method tabulateManifest()
Tabulate the concept set manifest
Arguments
filterCharacter. One of "active", "deleted", or "all". Defaults to "active".
tags_formatCharacter. One of "nested", "json", or "wide".
"nested" (default): Parse JSON tags into a nested tibble with tag_name/tag_value columns
"json": Keep tags as raw JSON string
"wide": Expand tags into individual columns (one per unique tag key)
Method viewManifest()
View the concept set manifest in RStudio viewer
Opens an interactive RStudio viewer showing key concept set metadata: id, label, category, tags, and file_path. This is a convenience function for exploring manifest contents without console clutter.
Method getAtlasConnection()
Method setAtlasConnection()
Arguments
atlasConnectionAn ATLAS connection object (from
getAtlasConnection()).
Method addConceptSetFile()
Register a local CIRCE JSON file in the manifest
Usage
ConceptSetManifest$addConceptSetFile(
filePath,
label,
category = "init",
tags = list()
)Arguments
filePathCharacter. Absolute or relative path to a valid CIRCE JSON file.
labelCharacter. Display name for the concept set.
categoryCharacter. Category for the concept set. Defaults to
"init".tagsNamed list. Optional extra metadata tags. Defaults to
list().
Method addAtlasConceptSet()
Fetch a single concept set from ATLAS and register it in the manifest
Usage
ConceptSetManifest$addAtlasConceptSet(
atlasId,
label,
category = "init",
tags = list(),
atlasConnection = NULL,
stopIfExists = TRUE
)Arguments
atlasIdInteger. The ATLAS concept set definition ID.
labelCharacter. Display name for the concept set.
categoryCharacter. Category for the concept set. Defaults to
"init".tagsNamed list. Optional extra metadata tags. Defaults to
list().atlasConnectionAn ATLAS connection object with a
getConceptSetDefinition(conceptSetId)method that returns a list withexpression(CIRCE JSON string) andsaveNameelements. IfNULL, falls back to the connection stored via$setAtlasConnection().stopIfExistsLogical. If TRUE (default), raises an error when an active concept set with this label is already registered. If FALSE, fetches the current definition from ATLAS and updates the registered concept set in place — same ID and file path, hash refreshed,
category/tags/atlasId replaced. An unchanged definition leaves the file untouched. Default: TRUE (fail-safe).
Method addCaprConceptSet()
Export a Capr ConceptSet to JSON and register it in the manifest
Usage
ConceptSetManifest$addCaprConceptSet(
caprConceptSet,
label,
category = "init",
tags = list(),
stopIfExists = TRUE
)Arguments
caprConceptSetA Capr
ConceptSetobject.labelCharacter. Display name for the concept set.
categoryCharacter. Category for the concept set. Defaults to
"init".tagsNamed list. Optional extra metadata tags. Defaults to
list().stopIfExistsLogical. If TRUE (default), raises an error when an active concept set with this label is already registered. If FALSE, updates the existing concept set in place via
updateCaprConceptSet()— it keeps its ID and file path, andcategory/tagsreplace the registered metadata (previous tags are dropped if none are supplied). Default: TRUE (fail-safe).
Method updateCaprConceptSet()
Update an existing Capr concept set's JSON definition
Takes a revised Capr ConceptSet object and upserts it over a concept set
already registered via addCaprConceptSet(): the JSON file recorded in
the manifest is overwritten in place and the manifest hash is refreshed,
so the concept set keeps its ID and file path. If the new definition is
identical to the registered one, nothing is changed.
Method importAtlasConceptSets()
Batch-import concept sets from ATLAS via a conceptSetsLoad dataframe
Either create a dataframe or read in a csv file with columns atlasId, label, category (required) plus any
additional columns treated as tag key-value pairs for tags. Calls addAtlasConceptSet() for each row inside
a tryCatch so a single failure does not abort the entire batch.
Usage
ConceptSetManifest$importAtlasConceptSets(
conceptSetsLoad,
atlasConnection = NULL,
stopIfExists = TRUE
)Arguments
conceptSetsLoada data frame requiring the columns atlasId, label and category used to bulk add cohorts to the manifest
atlasConnectionAn ATLAS connection object with a
getConceptSetDefinition(conceptSetId)method. IfNULL, falls back to the connection stored via$setAtlasConnection().stopIfExistsLogical. If TRUE (default), raises an error when any load row's atlasId is already registered in the manifest. If FALSE, those rows are updated in place instead (same ID/file path, hash refreshed, category/tags replaced) via
addAtlasConceptSet(stopIfExists = FALSE). Default: TRUE (fail-safe).
Method queryConceptSetsByIds()
Usage
ConceptSetManifest$queryConceptSetsByIds(
ids,
tags_format = c("nested", "json", "wide")
)Method queryConceptSetsByTag()
Arguments
tagStringsCharacter vector. One or more tags in the format "name: value" (e.g., "category: primary"). When multiple tags are supplied, the
matchargument controls whether a concept set must satisfy any or all of them.matchCharacter. "any" (default) returns concept sets matching at least one tag; "all" returns only concept sets matching every tag.
tags_formatCharacter. One of "nested", "json", or "wide".
"nested" (default): Tags as nested tibble with tag_name/tag_value columns
"json": Tags as raw JSON string
"wide": Tags expanded into individual columns
Method queryConceptSetsByCategory()
Arguments
categoryCharacter vector. One or more category to search for. A concept set is included when it matches at least one of the supplied category (OR logic).
matchTypeCharacter. Either "exact" for exact match or "pattern" for pattern matching. Defaults to "exact".
tags_formatCharacter. One of "nested", "json", or "wide".
"nested" (default): Tags as nested tibble with tag_name/tag_value columns
"json": Tags as raw JSON string
"wide": Tags expanded into individual columns
Method queryConceptSetsByTagName()
Usage
ConceptSetManifest$queryConceptSetsByTagName(
tagName,
tags_format = c("nested", "json", "wide")
)Method queryConceptSetsMissingTag()
Query concept sets missing a specific tag
Usage
ConceptSetManifest$queryConceptSetsMissingTag(
tagName,
tags_format = c("nested", "json", "wide")
)Method queryConceptSetsWithTagValues()
Query concept sets by tag value mapping
Usage
ConceptSetManifest$queryConceptSetsWithTagValues(
tagValueMapping,
tags_format = c("nested", "json", "wide")
)Arguments
tagValueMappingNamed list. Keys are tag names, values are tag values to match. Example:
list(status = "approved", type = "primary")requires both conditions (AND logic).tags_formatCharacter. One of "nested", "json", or "wide".
"nested" (default): Tags as nested tibble with tag_name/tag_value columns
"json": Tags as raw JSON string
"wide": Tags expanded into individual columns
Method queryConceptSetsByLabel()
Arguments
labelsCharacter vector. One or more labels to search for. A concept set is included when it matches at least one of the supplied labels (OR logic).
matchTypeCharacter. Either "exact" for exact match or "pattern" for pattern matching. Defaults to "exact".
tags_formatCharacter. One of "nested", "json", or "wide".
"nested" (default): Tags as nested tibble with tag_name/tag_value columns
"json": Tags as raw JSON string
"wide": Tags expanded into individual columns
Method getConceptSetsByTag()
Usage
ConceptSetManifest$getConceptSetsByTag(tagStrings, match = c("any", "all"))Arguments
tagStringsCharacter vector. One or more tags in the format "name: value" (e.g., "category: primary"). When multiple tags are supplied, the
matchargument controls whether a concept set must satisfy any or all of them.matchCharacter. "any" (default) returns concept sets matching at least one tag; "all" returns only concept sets matching every tag.
Method getConceptSetsByLabel()
Usage
ConceptSetManifest$getConceptSetsByLabel(
labels,
matchType = c("exact", "pattern")
)Method deleteConceptSet()
Soft delete a concept set (mark as deleted, preserve record)
Method combineConceptSets()
Combine multiple concept sets into a single unified concept set
Loads multiple concept sets from the manifest as Capr objects, merges them into a unified concept set using set logic (include, include+descendants, exclude, exclude+descendants), exports the result as JSON, and registers it in the manifest.
Usage
ConceptSetManifest$combineConceptSets(
conceptSetIds,
combinedLabel,
combinedCategory = "combined",
combinedTags = list(),
stopIfExists = TRUE
)Arguments
conceptSetIdsInteger vector. IDs of concept sets to combine (minimum 2).
combinedLabelCharacter. Display name for the combined concept set.
combinedCategoryCharacter. Category for the combined concept set. Defaults to
"combined".combinedTagsNamed list. Optional metadata tags for the combined set. Defaults to
list(). A tagsourceConceptSetIdsis automatically added with comma-separated source IDs.stopIfExistsLogical. If TRUE (default), raises an error when an active concept set with
combinedLabelis already registered. If FALSE, updates the existing concept set in place viaaddCaprConceptSet()'s upsert path — it keeps its ID and file path, andcombinedCategory/combinedTagsreplace the registered metadata. Default: TRUE (fail-safe).
Method updateConceptSetLabel()
Update a concept set label
Method updateConceptSetCategory()
Update a concept set category
Method updateConceptSetTags()
Update concept set tags
Method removeConceptSetTag()
Remove a specific tag from a concept set
Method modifyConceptSetTagValue()
Modify the value of an existing tag
Method addConceptSetTag()
Add a single tag to a concept set (non-destructive)
Method mergeTagsIntoConceptSet()
Merge multiple tags into a concept set (non-destructive, additive)
Method getTagValue()
Get value of a single tag for a concept set
Method renameTagKey()
Rename a tag key across specified concept sets (or all concept sets)
Method bulkModifyTagValue()
Bulk modify a tag value across concept sets matching an old value
Method checkAtlasConceptSets()
Auto-detect changes to ATLAS concept sets in remote repository
Queries the manifest for all active ATLAS concept sets (identified by atlasId in tags),
fetches their current definitions from ATLAS, computes hashes, and compares against
the stored local hash. Provides a read-only summary of which concept sets have changed
in ATLAS since import. No modifications are made.
Arguments
atlasConnectionAn ATLAS connection object with a method
getConceptSetDefinition(conceptSetId)that returns a list with anexpressionelement (the CIRCE JSON as a string). IfNULL(default), uses the connection stored via$setAtlasConnection(). If no connection is available, raises an error.
Returns
Invisible tibble with columns:
id: Concept set ID in the local manifestlabel: Concept set labelatlasId: ATLAS concept set IDfilePath: Local path to the JSON filehasChanged: Logical, TRUE if remote definition differs from local hashlocalHash: Hash of the stored JSON fileremoteHash: Hash of the current ATLAS definition
Method updateAtlasConceptSets()
Update ATLAS concept sets with remote definitions
Fetches current definitions from ATLAS for concept sets that have changed and updates the stored JSON files and manifest entries. This is the modification phase that applies changes detected by checkAtlasConceptSets().
Method cleanupMissing()
Clean up missing concept sets from manifest
Method syncManifest()
Scans the json/ subdirectory of the concept sets folder, reconciles it against
the SQLite manifest, and updates both the database and the in-memory list:
Active manifest records whose file no longer exists are soft-deleted.
Existing files whose JSON hash has changed are updated in the manifest.
Orphaned files on disk not in manifest are automatically deleted.
Stored file paths are resolved against the study repository root before any
disk comparison, and hashes are computed from file contents only, so a row
is reported as "hash_updated" only when the file's contents actually
changed — never because its stored path used an older convention. To
rewrite legacy stored paths to the current repo-root-relative form, run
normalizeConceptSetManifestPaths() once.
Method grabConceptInfoFromSet()
Retrieve concept information for all concepts in a concept set
Fetches the standard concepts included in a concept set from the OMOP vocabulary tables. The concept set definition (stored as CIRCE JSON) is used to build a query that retrieves all concept IDs and names matching the set definition. Results are returned as a tibble with concept identifiers and display names.
Method extractSourceCodes()
Finds source codes from specified vocabularies that map to each concept set's standard concepts. Results are exported to a single xlsx file with one sheet per concept set, saved in the inputs/conceptSets folder. The function provides interactive vocabulary suggestions based on detected concept set domains.
Arguments
sourceVocabsCharacter vector. Source vocabulary IDs to search for. Valid options: "ICD9CM", "ICD10CM", "HCPCS", "CPT4", "LOINC", "NDC". Defaults to c("ICD10CM"). The function will suggest appropriate vocabularies based on the domains of your concept sets and prompt you to use them.
outputFolderCharacter. Path where the xlsx file will be saved. Defaults to "inputs/conceptSets".
Returns
Invisibly returns NULL. Saves xlsx file to outputFolder and prints status messages via cli package. Output file is ready to open in Excel or other spreadsheet software.
Extract Included Standard Concepts for Concept Sets
Finds standard concepts that are included in (map TO) each concept set's included concepts. Results are exported to a single xlsx file with one sheet per concept set, saved in the inputs/conceptSets folder.
Method extractIncludedCodes()
Usage
ConceptSetManifest$extractIncludedCodes(
outputFolder = here::here("inputs/conceptSets")
)