Evaluation of the hierarchy attribute extraction pipeline¶
Here we evaluate the performance of Step 2 of the Ariadne pipeline: extracting SNOMED CT hierarchy attributes for medical terms.
The pipeline:
- Retrieves similar reference SNOMED terms (few-shot examples)
- Uses an LLM to extract attribute components from the medical term
- Retrieves candidate concepts for each extracted component via vector search
- Uses an LLM to select the best candidate for each attribute
Setup¶
Before running this notebook, make sure the environment is set up as described in the README. This includes:
- Credentials for a database with the OHDSI Vocabulary loaded (including
snomed_attributetable) - Credentials for the LLM API
- Java 17+ installed and on PATH
- snomed-owl-toolkit JAR — download from GitHub releases
- SNOMED CT International Edition RF2 snapshot — obtain from MLDS
All results will be stored in the data/notebook_results folder.
Most code blocks will load results from file if they already exist, to save time and costs.
Delete those files to rerun the corresponding steps.
import json
from pathlib import Path
import logging
import pandas as pd
from dotenv import load_dotenv
from ariadne.utils.config import Config
project_root = Path.cwd().parent.parent
load_dotenv()
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
config = Config()
Create Attribute table¶
We require the snomed_attribute table in the database. This table contains all SNOMED concepts that can be retrieved as attribute values (e.g. finding sites, morphologies, causative agents). Use if_exists = skip to use existing tables and if_exists = rebuild to recreate the tables.
from ariadne.hierarchy.attribute_table_builder import build_attribute_table
build_attribute_table(
config.hierarchy,
if_exists="skip"
)
2026-10-28 23:03:55,259 snomed_attribute already populated — skipping.
Gold standard¶
We use the hierarchy attributes gold standard file. Each row maps a source concept to an expected SNOMED attribute (e.g. finding site, associated morphology).
This gold standard contains the following columns:
source_concept_id: the OMOP concept ID of the source term (may be empty for new vocabularies)source_concept_code: the source concept codesource_vocabulary_id: the source vocabulary (together withsource_concept_codeforms the primary key)source_term: the source term texttarget_concept_id: the concept ID of the expected attribute value (e.g. a finding site concept)target_concept_code: the concept code of the expected attribute value (e.g. a finding site concept)target_concept_name: the name of the expected attribute valueattribute_category: the SNOMED relationship type (e.g.Has finding site,Has asso morph,Has causative agent)
attribute_gs_path = project_root / "data" / "gold_standards" / "hierarchy_attributes_snomed_gs.csv"
# Alternative: use unmatched terms from the exact matching pipeline (Step 1)
# attribute_gs_path = project_root / "data" / "notebook_results" / "exact_matching_vector_search_results.csv"
attribute_gs = pd.read_csv(attribute_gs_path)
# If using exact_matching_vector_search_results.csv, filter to unmatched terms only:
# attribute_gs = attribute_gs[attribute_gs["mapped_concept_id"] == -1]
print(f"Attribute GS: {len(attribute_gs)} rows")
unique_terms = attribute_gs[
["source_concept_id", "source_concept_code", "source_vocabulary_id", "source_term"]
].drop_duplicates()
print(f"Unique terms: {len(unique_terms)}")
attribute_gs.head(10)
Attribute GS: 776 rows Unique terms: 275
| source_concept_id | source_concept_code | source_vocabulary_id | source_term | target_concept_id | target_concept_code | target_concept_name | attribute_category | |
|---|---|---|---|---|---|---|---|---|
| 0 | 22274 | 94899001 | SNOMED | Neoplasm of uncertain behavior of larynx | 4262229 | 4596009 | Laryngeal structure | Has finding site |
| 1 | 22274 | 94899001 | SNOMED | Neoplasm of uncertain behavior of larynx | 4313421 | 86251006 | Neoplasm of uncertain behavior | Has asso morph |
| 2 | 22288 | 191169008 | SNOMED | Hereditary elliptocytosis | 4030871 | 14089001 | Red blood cell count | Has interprets |
| 3 | 22288 | 191169008 | SNOMED | Hereditary elliptocytosis | 4083207 | 281300000 | Below reference range | Has interpretation |
| 4 | 22288 | 191169008 | SNOMED | Hereditary elliptocytosis | 4116830 | 255399007 | Congenital | Has occurrence |
| 5 | 22288 | 191169008 | SNOMED | Hereditary elliptocytosis | 4181412 | 52101004 | Present | Has interpretation |
| 6 | 22288 | 191169008 | SNOMED | Hereditary elliptocytosis | 4204670 | 308490002 | Pathological developmental process | Has pathology |
| 7 | 22288 | 191169008 | SNOMED | Hereditary elliptocytosis | 4218976 | 41898006 | Erythrocyte | Has finding site |
| 8 | 22288 | 191169008 | SNOMED | Hereditary elliptocytosis | 4224294 | 404227002 | Hemolysis | Has interprets |
| 9 | 22288 | 191169008 | SNOMED | Hereditary elliptocytosis | 4259404 | 45028007 | Elliptocyte | Has asso morph |
Run the hierarchy attribute extraction pipeline¶
We use process_hierarchy from the runner module, which:
- Iterates over each unique term in the gold standard
- Runs the four-step attribute extraction pipeline (
extract_attributes) - Checks attributes against SNOMED MRCM rules
- Caches each successful result to a per-term JSON file, so re-running resumes and only regenerates terms that failed or were never processed
from ariadne.hierarchy.llm_attribute_runner import process_hierarchy
from ariadne.evaluation.hierarchy_attribute_evaluator import build_prediction_rows
from ariadne.hierarchy.searchers import SnomedAttributeSearcher, SnomedReferenceConceptVectorSearcher
from ariadne.hierarchy.mrcm import MrcmRules
from ariadne.hierarchy.mrcm_validator import validate_role_groups
# Per-term cache in data/hierarchy_attribute_responses makes re-runs free; delete it after prompt changes.
gs_concept_ids = set(attribute_gs["source_concept_id"].dropna().unique())
attr_searcher = SnomedAttributeSearcher(hierarchy_settings=config.hierarchy)
ref_searcher = SnomedReferenceConceptVectorSearcher(hierarchy_settings=config.hierarchy, exclude_concept_ids=gs_concept_ids)
results = process_hierarchy(
unique_terms,
attribute_searcher=attr_searcher,
reference_searcher=ref_searcher,
hierarchy_settings=config.hierarchy,
source_term_column="source_term",
source_code_column="source_concept_code",
source_vocab_column="source_vocabulary_id",
source_id_column="source_concept_id",
)
with open(project_root / "data" / "notebook_results" / "hierarchy_results_raw.json", "w") as f:
json.dump(results, f, indent=2, default=str) # debugging copy only
total_cost = sum(r.get("cost", {}).get("total_cost", 0.0) for r in results if "cost" in r)
print(f"Total API cost: ${total_cost:.4f}")
mrcm_rules = MrcmRules.load(config.hierarchy.mrcm, config.hierarchy.attributes)
violations = validate_role_groups(results, mrcm_rules)
violations.to_csv(project_root / "data" / "notebook_results" / "hierarchy_mrcm_violations.csv", index=False)
print(f"MRCM violations: {len(violations)} across {violations['source_term'].nunique() if len(violations) else 0} terms")
results_df = pd.DataFrame(build_prediction_rows(results, config.hierarchy))
results_df.to_csv(project_root / "data" / "notebook_results" / "hierarchy_attribute_results.csv", index=False)
print(f"Saved {len(results_df)} attribute rows to hierarchy_attribute_results.csv")
results_df.head()
2026-10-28 23:03:59,948 Resuming: 275 terms already cached in /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/data/hierarchy_attribute_responses 2026-10-28 23:03:59,956 All terms already processed.
Total API cost: $6.6876 MRCM violations: 0 across 0 terms Saved 765 attribute rows to hierarchy_attribute_results.csv
| source_key | source_concept_id | source_concept_code | source_vocabulary_id | source_term | predicted_target_concept_id | predicted_target_concept_name | predicted_target_concept_code | attribute_category | relationship_groups | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | SNOMED:10155006 | 73021 | 10155006 | SNOMED | Bilateral congenital dislocation of hip | 4227147 | Dislocation | 87642003 | Has asso morph | 1;2 |
| 1 | SNOMED:10155006 | 73021 | 10155006 | SNOMED | Bilateral congenital dislocation of hip | 4177897 | Right hip joint structure | 362908009 | Has finding site | 1 |
| 2 | SNOMED:10155006 | 73021 | 10155006 | SNOMED | Bilateral congenital dislocation of hip | 4176631 | Left hip joint structure | 362907004 | Has finding site | 2 |
| 3 | SNOMED:10155006 | 73021 | 10155006 | SNOMED | Bilateral congenital dislocation of hip | 4204670 | Pathological developmental process | 308490002 | Has pathology | 1;2 |
| 4 | SNOMED:10155006 | 73021 | 10155006 | SNOMED | Bilateral congenital dislocation of hip | 4116830 | Congenital | 255399007 | Has occurrence | 1;2 |
Evaluation¶
We evaluate the pipeline results against the gold standard using a full outer join.
Each predicted attribute is matched against the expected attribute by (source_key, target_concept_id, attribute_category),
where source_key combines source_vocabulary_id and source_concept_code.
As of now, excludes comparison of groups.
from ariadne.evaluation.hierarchy_attribute_evaluator import evaluate_results
# Set output directory relative to project root
output_dir = project_root / "data" / "notebook_results"
eval_df = evaluate_results(results, str(attribute_gs_path), output_dir, config.hierarchy)
eval_df.head()
2026-10-28 23:04:00,213 Gold standard rows: 778 2026-10-28 23:04:00,214 Predicted rows: 765 2026-10-28 23:04:00,214 Matched: 702 2026-10-28 23:04:00,214 Precision: 91.8% 2026-10-28 23:04:00,215 Recall: 90.2% 2026-10-28 23:04:00,215 F1: 91.0% 2026-10-28 23:04:00,222 Combined evaluation saved: /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/data/notebook_results/hierarchy_attribute_evaluation.csv (841 rows)
| source_key | source_concept_id | source_concept_code | source_vocabulary_id | source_term | attribute_category | gs_target_concept_id | gs_target_concept_code | gs_target_concept_name | predicted_target_concept_id | predicted_target_concept_code | predicted_target_concept_name | matched | status | relationship_groups | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | SNOMED:10155006 | 73021.0 | 10155006.0 | SNOMED | Bilateral congenital dislocation of hip | Has asso morph | 4227147.0 | 87642003.0 | Dislocation | 4227147 | 87642003 | Dislocation | True | match | 1;2 |
| 1 | SNOMED:10155006 | 73021.0 | 10155006.0 | SNOMED | Bilateral congenital dislocation of hip | Has finding site | 4176631.0 | 362907004.0 | Left hip joint structure | 4176631 | 362907004 | Left hip joint structure | True | match | 2 |
| 2 | SNOMED:10155006 | 73021.0 | 10155006.0 | SNOMED | Bilateral congenital dislocation of hip | Has finding site | 4177897.0 | 362908009.0 | Right hip joint structure | 4177897 | 362908009 | Right hip joint structure | True | match | 1 |
| 3 | SNOMED:10155006 | 73021.0 | 10155006.0 | SNOMED | Bilateral congenital dislocation of hip | Has occurrence | 4116830.0 | 255399007.0 | Congenital | 4116830 | 255399007 | Congenital | True | match | 1;2 |
| 4 | SNOMED:10155006 | 73021.0 | 10155006.0 | SNOMED | Bilateral congenital dislocation of hip | Has pathology | 4204670.0 | 308490002.0 | Pathological developmental process | 4204670 | 308490002 | Pathological developmental process | True | match | 1;2 |
Per-category breakdown¶
breakdown = (
eval_df.groupby("attribute_category")["status"]
.value_counts()
.unstack(fill_value=0)
.reindex(columns=["match", "missed", "extra"], fill_value=0)
)
breakdown["precision%"] = (
breakdown["match"] / (breakdown["match"] + breakdown["extra"]).clip(lower=1) * 100
).round(1)
breakdown["recall%"] = (
breakdown["match"] / (breakdown["match"] + breakdown["missed"]).clip(lower=1) * 100
).round(1)
breakdown
| status | match | missed | extra | precision% | recall% |
|---|---|---|---|---|---|
| attribute_category | |||||
| During | 2 | 0 | 0 | 100.0 | 100.0 |
| Finding asso with | 3 | 0 | 0 | 100.0 | 100.0 |
| Has asso finding | 0 | 0 | 1 | 0.0 | 0.0 |
| Has asso morph | 225 | 14 | 13 | 94.5 | 94.1 |
| Has causative agent | 18 | 1 | 2 | 90.0 | 94.7 |
| Has clinical course | 28 | 0 | 2 | 93.3 | 100.0 |
| Has due to | 49 | 4 | 0 | 100.0 | 92.5 |
| Has finding context | 2 | 0 | 0 | 100.0 | 100.0 |
| Has finding site | 246 | 42 | 39 | 86.3 | 85.4 |
| Has interpretation | 17 | 3 | 3 | 85.0 | 85.0 |
| Has interprets | 18 | 6 | 2 | 90.0 | 75.0 |
| Has occurrence | 27 | 1 | 0 | 100.0 | 96.4 |
| Has pathology | 52 | 3 | 1 | 98.1 | 94.5 |
| Has relat context | 2 | 0 | 0 | 100.0 | 100.0 |
| Has severity | 2 | 0 | 0 | 100.0 | 100.0 |
| Has temporal context | 2 | 1 | 0 | 100.0 | 66.7 |
| Occurs after | 9 | 1 | 0 | 100.0 | 90.0 |
Stated parent selection¶
We then find most appropriate primitive parent (N set by 'max_parents' in yaml).
For each term, we find similar standard SNOMED concepts (num_neighbours) and retrieve their primitive parents. A parent qualifies when has at least min_support (sum of similarities of neighbours sitting below a candidate parents over total similarity). Then, the deepest qualifying primitives (min_levels_of_separation from Clinical Finding) are chosen. When nothing qualifies, the term falls back to Clinical Finding.
For held-out SNOMED terms the source concept and its descendants are excluded from the neighbours (exclude_source_descendants).
import psycopg
from ariadne.hierarchy.parent_selector import build_stated_parents_map
from ariadne.utils.utils import get_environment_variable
conn_str = get_environment_variable("VOCAB_CONNECTION_STRING").replace("+psycopg2", "").replace("+psycopg", "")
schema = get_environment_variable("VOCAB_SCHEMA")
parent_settings = config.hierarchy.parent_selection
with open(project_root / "data" / "notebook_results" / "hierarchy_results_raw.json") as fh:
raw_results = json.load(fh)
with psycopg.connect(conn_str) as parent_conn:
stated_parents_map, parent_details = build_stated_parents_map(raw_results, parent_conn, schema, parent_settings)
parent_details.to_csv(project_root / "data" / "notebook_results" / "hierarchy_stated_parents.csv", index=False)
per_source = parent_details.drop_duplicates("source_concept_id")
print(f"Stated parents (num_neighbours={parent_settings.num_neighbours}, min_support={parent_settings.min_support}, "
f"max_parents={parent_settings.max_parents}):")
print(f" Primitive parent below Clinical finding : {(~per_source['is_fallback']).sum()}/{len(per_source)}")
print(f" Fell back to Clinical finding : {per_source['is_fallback'].sum()}/{len(per_source)}")
parent_details.head()
2026-10-28 23:04:02,651 Stated parents: 258 terms with a primitive parent below the root, 17 fell back to Clinical finding.
Stated parents (num_neighbours=10, min_support=0.5, max_parents=2): Primitive parent below Clinical finding : 258/275 Fell back to Clinical finding : 17/275
| source_concept_id | source_term | parent_rank | parent_concept_id | parent_concept_code | parent_concept_name | parent_depth | support | num_neighbours | is_fallback | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 73021 | Bilateral congenital dislocation of hip | 1 | 4274025 | 64572001 | Disease | 1 | 1.0000 | 10 | False |
| 1 | 26378 | Hyperpituitarism | 1 | 4274025 | 64572001 | Disease | 1 | 1.0000 | 10 | False |
| 2 | 25572 | Disorder of salivary gland | 1 | 4274025 | 64572001 | Disease | 1 | 0.8102 | 10 | False |
| 3 | 28356 | Overlapping malignant neoplasm of major saliva... | 1 | 4274025 | 64572001 | Disease | 1 | 1.0000 | 10 | False |
| 4 | 25748 | Overlapping malignant neoplasm of esophagus | 1 | 4274025 | 64572001 | Disease | 1 | 1.0000 | 10 | False |
# Evaluation: stated parents vs. the held-out concepts' proximal primitive parents fetched from OHDSI Vocabularies
from ariadne.evaluation.stated_parent_evaluator import OUTCOMES, evaluate_stated_parents
with psycopg.connect(conn_str) as parent_conn:
parent_eval = evaluate_stated_parents(parent_details, parent_conn, schema, parent_settings)
parent_eval.to_csv(project_root / "data" / "notebook_results" / "hierarchy_stated_parent_evaluation.csv", index=False)
counts = parent_eval["outcome"].value_counts().reindex(OUTCOMES, fill_value=0)
print(pd.DataFrame({"count": counts, "%": (counts / len(parent_eval) * 100).round(1)}).to_string())
parent_eval[parent_eval["outcome"] == "wrong"].head()
count % outcome exact 199 72.4 partial 19 6.9 too_general 40 14.5 wrong 14 5.1 no_reference 3 1.1
| source_concept_id | source_term | selected_parents | reference_parents | outcome | |
|---|---|---|---|---|---|
| 26 | 609115 | Proliferative retinopathy of left eye due to d... | Nonproliferative retinopathy due to diabetes m... | Proliferative retinopathy | wrong |
| 36 | 26908 | Vincent's angina | Non-plaque induced gingival disease; Traumatic... | Traumatic or non-traumatic injury; Sputum finding | wrong |
| 42 | 72857 | Late effect of rickets | Rickets | Disease | wrong |
| 43 | 22288 | Hereditary elliptocytosis | Anemia due to intrinsic red cell abnormality; ... | Hereditary hemolytic anemia; Erythrocyte membr... | wrong |
| 63 | 74739 | Osteopathy from poliomyelitis | Neuromuscular scoliosis | Disease | wrong |
RF2 Delta Export¶
Convert the predicted attributes to SNOMED CT RF2 delta files for use with the ELK reasoner.
For SNOMED held-out set, source OMOP concept IDs are remapped to synthetic SCTIDs (starting at 1 000 000 001).
Concept definitions are expressed as OWL Functional Syntax axioms in the OWL Axiom Reference Set:
- A concept with attributes becomes
EquivalentClasses(:src ObjectIntersectionOf(:parent ObjectSomeValuesFrom(:609096000 ...))) - A concept with no attributes falls back to
SubClassOf(:src :parent)
The output ZIP contains files under Delta/:
Delta/Terminology/sct2_Concept_Delta_INT_{date}.txt— one row per source term (sufficiently defined)Delta/Terminology/sct2_StatedRelationship_Delta_INT_{date}.txt— header-only placeholderDelta/Terminology/sct2_Relationship_Delta_INT_{date}.txt— header-only placeholder (required by toolkit)Delta/Refset/Content/der2_sRefset_OWLAxiomDelta_INT_{date}.txt— OWL axiom per source conceptDelta/Refset/Metadata/der2_ssRefset_ModuleDependencyDelta_INT_{date}.txt— module dependency
import sys, shutil
# Force a fresh import — drop ALL ariadne.hierarchy modules from cache
for mod_name in list(sys.modules.keys()):
if mod_name.startswith("ariadne"):
del sys.modules[mod_name]
from ariadne.hierarchy.rf2_exporter import export_to_rf2
rf2_output_dir = project_root / "data" / "rf2_output"
# Clean previous delta files to avoid stale artifacts
old_delta = rf2_output_dir / "SnomedCT_YourDelta"
if old_delta.exists():
shutil.rmtree(old_delta)
old_delta2 = rf2_output_dir / "Delta"
if old_delta2.exists():
shutil.rmtree(old_delta2)
zip_path, id_map = export_to_rf2(
source=project_root / "data" / "notebook_results" / "hierarchy_attribute_results.csv",
output_dir=rf2_output_dir,
hierarchy_settings=config.hierarchy,
stated_parent=stated_parents_map,
)
print(f"RF2 delta written to: {zip_path}")
# Verify OWL content inline
import zipfile
with zipfile.ZipFile(zip_path) as zf:
owl_name = next(n for n in zf.namelist() if "OWLAxiom" in n)
with zf.open(owl_name) as f:
owl_lines = f.read().decode("utf-8").splitlines()
print(f"OWL axiom rows: {len(owl_lines) - 1}")
if owl_lines[1:]:
sample = owl_lines[1].split("\t")
print(f"Sample expression: {sample[-1][:150]}")
id_map.head()
2026-10-28 23:04:05,183 Exporting 765 attribute rows to RF2 delta (date=20261028)… 2026-10-28 23:04:05,189 ID mapping: 272 concepts → id_mapping_20261028.csv 2026-10-28 23:04:05,190 Concept file: 272 concepts → sct2_Concept_Delta_INT_20261028.txt 2026-10-28 23:04:05,192 StatedRelationship file (header-only): sct2_StatedRelationship_Delta_INT_20261028.txt 2026-10-28 23:04:05,192 Relationship file (header-only placeholder): sct2_Relationship_Delta_INT_20261028.txt 2026-10-28 23:04:05,196 OWL Axiom refset: 272 concepts (272 with attributes, 0 attr-only Is a) → der2_sRefset_OWLAxiomDelta_INT_20261028.txt 2026-10-28 23:04:05,199 Module Dependency refset: der2_ssRefset_ModuleDependencyDelta_INT_20261028.txt 2026-10-28 23:04:05,208 RF2 delta ZIP: /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/data/rf2_output/snomed_delta_20261028.zip
RF2 delta written to: /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/data/rf2_output/snomed_delta_20261028.zip OWL axiom rows: 272 Sample expression: EquivalentClasses(:1000000001 ObjectIntersectionOf(:64572001 ObjectSomeValuesFrom(:609096000 ObjectIntersectionOf(ObjectSomeValuesFrom(:116676008 :876
| omop_concept_id | synthetic_sctid | concept_name | |
|---|---|---|---|
| 0 | 73021 | 1000000001 | Bilateral congenital dislocation of hip |
| 1 | 26378 | 1000000002 | Hyperpituitarism |
| 2 | 25572 | 1000000003 | Disorder of salivary gland |
| 3 | 28356 | 1000000004 | Overlapping malignant neoplasm of major saliva... |
| 4 | 25748 | 1000000005 | Overlapping malignant neoplasm of esophagus |
SNOMED Classification (ELK Reasoner)¶
Run classifier (snomed-owl-toolkit) that takes the RF2 delta ZIP produced above and a base SNOMED CT snapshot, converts both to OWL, runs ELK, and returns the inferred parent relationships.
Pre-classification validation¶
Check the RF2 delta for structural issues before sending it to the classifier.
from ariadne.hierarchy.classifier import (
classify_delta,
parse_classification_results,
pre_classification_checks
)
delta_zip = project_root / "data" / "rf2_output" / next(
f.name for f in (project_root / "data" / "rf2_output").iterdir()
if f.name.startswith("snomed_delta_") and f.name.endswith(".zip")
)
issues = pre_classification_checks(delta_zip, config.hierarchy)
if issues:
print("Issues found:")
for issue in issues:
print(f" - {issue}")
else:
print("All pre-classification checks passed.")
All pre-classification checks passed.
Run classification¶
The snomed-owl-toolkit converts RF2 → OWL, runs ELK, and produces a results ZIP with the inferred "Is a" relationships.
results_zip = classify_delta(
delta_zip,
base_snomed_zip=str(project_root / "data" / "SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20260301T120000Z.zip"),
toolkit_jar=str(project_root / "tools" / "snomed-owl-toolkit-5.3.0-executable.jar"),
output_dir=project_root / "data" / "rf2_output",
)
print(f"Classification results: {results_zip}")
2026-10-28 23:04:05,234 Running classification: java -Xms4g --add-opens java.base/java.lang=ALL-UNNAMED -jar /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/tools/snomed-owl-toolkit-5.3.0-executable.jar -classify -rf2-snapshot-archives /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/data/SnomedCT_ManagedServiceUS_PRODUCTION_US1000124_20260301T120000Z.zip -rf2-authoring-delta-archive /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/data/rf2_output/snomed_delta_20261028.zip
2026-10-28 23:05:29,351 snomed-owl-toolkit stdout:
2026-10-28 23:04:05,724 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: started
2026-10-28 23:04:05,724 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - Checking requested reasoner is available
2026-10-28 23:04:05,726 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: Create reasoner factory took 0.003 seconds
2026-10-28 23:04:05,726 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - Building snomedTaxonomy
2026-10-28 23:04:07,576 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._Snapshot
2026-10-28 23:04:07,577 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._Refset
2026-10-28 23:04:07,578 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/.DS_Store
2026-10-28 23:04:07,579 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._.DS_Store
2026-10-28 23:04:07,580 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._Terminology
2026-10-28 23:04:11,413 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/.DS_Store
2026-10-28 23:04:11,413 [ERROR ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Failed to create file /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/.DS_Store
2026-10-28 23:04:11,416 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._.DS_Store
2026-10-28 23:04:11,416 [ERROR ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Failed to create file /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._.DS_Store
2026-10-28 23:04:11,417 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._Content
2026-10-28 23:04:11,418 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._Language
2026-10-28 23:04:11,419 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._Map
2026-10-28 23:04:11,428 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._Metadata
2026-10-28 23:04:11,432 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_Identifier_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:11,446 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._sct2_Identifier_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:11,461 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_Description_Snapshot-en_US1000124_20260301.txt
2026-10-28 23:04:12,718 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._sct2_Description_Snapshot-en_US1000124_20260301.txt
2026-10-28 23:04:12,720 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_sRefset_OWLExpressionSnapshot_US1000124_20260301.txt
2026-10-28 23:04:13,466 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._sct2_sRefset_OWLExpressionSnapshot_US1000124_20260301.txt
2026-10-28 23:04:13,467 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_Relationship_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:15,061 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._sct2_Relationship_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:15,062 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_StatedRelationship_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:15,551 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._sct2_StatedRelationship_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:15,552 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_RelationshipConcreteValues_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:15,570 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._sct2_RelationshipConcreteValues_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:15,571 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_Concept_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:15,681 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._sct2_Concept_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:15,682 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_TextDefinition_Snapshot-en_US1000124_20260301.txt
2026-10-28 23:04:15,742 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._sct2_TextDefinition_Snapshot-en_US1000124_20260301.txt
2026-10-28 23:04:19,601 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_Refset_SimpleSnapshot_US1000124_20260301.txt
2026-10-28 23:04:19,623 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_Refset_SimpleSnapshot_US1000124_20260301.txt
2026-10-28 23:04:19,625 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cRefset_AttributeValueSnapshot_US1000124_20260301.txt
2026-10-28 23:04:20,304 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_cRefset_AttributeValueSnapshot_US1000124_20260301.txt
2026-10-28 23:04:20,305 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cRefset_AssociationSnapshot_US1000124_20260301.txt
2026-10-28 23:04:20,541 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_cRefset_AssociationSnapshot_US1000124_20260301.txt
2026-10-28 23:04:20,542 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cRefset_LanguageSnapshot-en_US1000124_20260301.txt
2026-10-28 23:04:23,315 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_cRefset_LanguageSnapshot-en_US1000124_20260301.txt
2026-10-28 23:04:23,316 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_iisssccRefset_ExtendedMapSnapshot_US1000124_20260301.txt
2026-10-28 23:04:24,560 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_iisssccRefset_ExtendedMapSnapshot_US1000124_20260301.txt
2026-10-28 23:04:24,561 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_sRefset_SimpleMapSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,131 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_sRefset_SimpleMapSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,132 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_ssRefset_ModuleDependencySnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,133 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_ssRefset_ModuleDependencySnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,134 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_scsRefset_ComponentAnnotationStringValueSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,140 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_scsRefset_ComponentAnnotationStringValueSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,142 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_sscsRefset_MemberAnnotationStringValueSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,143 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_sscsRefset_MemberAnnotationStringValueSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,144 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_sssssssRefset_MRCMDomainSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,146 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_sssssssRefset_MRCMDomainSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,147 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cissccRefset_MRCMAttributeDomainSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,149 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_cissccRefset_MRCMAttributeDomainSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,150 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cciRefset_RefsetDescriptorSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,151 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_cciRefset_RefsetDescriptorSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,152 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_ciRefset_DescriptionTypeSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,153 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_ciRefset_DescriptionTypeSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,157 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_ssccRefset_MRCMAttributeRangeSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,159 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_ssccRefset_MRCMAttributeRangeSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,160 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cRefset_MRCMModuleScopeSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,160 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/._der2_cRefset_MRCMModuleScopeSnapshot_US1000124_20260301.txt
2026-10-28 23:04:25,194 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Loading SNAPSHOT release files ReleaseFiles{conceptPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_Concept_Snapshot_US1000124_20260301.txt], descriptionPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_Description_Snapshot-en_US1000124_20260301.txt], textDefinitionPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_TextDefinition_Snapshot-en_US1000124_20260301.txt], relationshipPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_Relationship_Snapshot_US1000124_20260301.txt], concreteRelationshipPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_RelationshipConcreteValues_Snapshot_US1000124_20260301.txt], statedRelationshipPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_StatedRelationship_Snapshot_US1000124_20260301.txt], identifierPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_Identifier_Snapshot_US1000124_20260301.txt], refsetPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_ssRefset_ModuleDependencySnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_iisssccRefset_ExtendedMapSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_Refset_SimpleSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cRefset_AttributeValueSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_scsRefset_ComponentAnnotationStringValueSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_sscsRefset_MemberAnnotationStringValueSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_sRefset_SimpleMapSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_sssssssRefset_MRCMDomainSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/sct2_sRefset_OWLExpressionSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cissccRefset_MRCMAttributeDomainSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cciRefset_RefsetDescriptorSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cRefset_LanguageSnapshot-en_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_ciRefset_DescriptionTypeSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_ssccRefset_MRCMAttributeRangeSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cRefset_AssociationSnapshot_US1000124_20260301.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip3270631437228044203/1/der2_cRefset_MRCMModuleScopeSnapshot_US1000124_20260301.txt]}
2026-10-28 23:04:25,194 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Gathering effective dates for effective component filtering.
2026-10-28 23:04:25,199 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading concepts
2026-10-28 23:04:25,766 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 537781 concepts read from sct2_Concept_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:25,768 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - includedReferenceSetPathPatterns: [.*_sRefset_OWL.*, .*_cissccRefset_MRCMAttributeDomain.*]
2026-10-28 23:04:25,769 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - refset 'sct2_sRefset_OWLExpressionSnapshot_US1000124_20260301.txt' matches pattern '.*_sRefset_OWL.*'
2026-10-28 23:04:25,770 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - refset 'der2_cissccRefset_MRCMAttributeDomainSnapshot_US1000124_20260301.txt' matches pattern '.*_cissccRefset_MRCMAttributeDomain.*'
2026-10-28 23:04:25,770 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading relationships
2026-10-28 23:04:29,997 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 3614587 relationships read from sct2_Relationship_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:29,998 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading concrete relationships
2026-10-28 23:04:30,037 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 37332 concrete relationships read from sct2_RelationshipConcreteValues_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:30,037 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading identifiers
2026-10-28 23:04:30,038 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 0 identifiers read from sct2_Identifier_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:30,038 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading relationships
2026-10-28 23:04:31,517 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 1050121 relationships read from sct2_StatedRelationship_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:31,518 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading reference set members
2026-10-28 23:04:32,230 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 419691 reference set members read from sct2_sRefset_OWLExpressionSnapshot_US1000124_20260301.txt
2026-10-28 23:04:32,231 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading reference set members
2026-10-28 23:04:32,232 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 172 reference set members read from der2_cissccRefset_MRCMAttributeDomainSnapshot_US1000124_20260301.txt
2026-10-28 23:04:32,233 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading concepts
2026-10-28 23:04:32,798 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 537781 concepts read from sct2_Concept_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:32,798 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - includedReferenceSetPathPatterns: [.*_sRefset_OWL.*, .*_cissccRefset_MRCMAttributeDomain.*]
2026-10-28 23:04:32,798 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - refset 'sct2_sRefset_OWLExpressionSnapshot_US1000124_20260301.txt' matches pattern '.*_sRefset_OWL.*'
2026-10-28 23:04:32,799 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - refset 'der2_cissccRefset_MRCMAttributeDomainSnapshot_US1000124_20260301.txt' matches pattern '.*_cissccRefset_MRCMAttributeDomain.*'
2026-10-28 23:04:32,799 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading relationships
2026-10-28 23:04:37,937 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 3614587 relationships read from sct2_Relationship_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:37,938 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading concrete relationships
2026-10-28 23:04:38,038 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 37332 concrete relationships read from sct2_RelationshipConcreteValues_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:38,038 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading identifiers
2026-10-28 23:04:38,038 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 0 identifiers read from sct2_Identifier_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:38,038 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading relationships
2026-10-28 23:04:39,338 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 1050121 relationships read from sct2_StatedRelationship_Snapshot_US1000124_20260301.txt
2026-10-28 23:04:39,338 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading reference set members
2026-10-28 23:04:39,775 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 10,000 axioms...
2026-10-28 23:04:40,074 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 20,000 axioms...
2026-10-28 23:04:40,340 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 30,000 axioms...
2026-10-28 23:04:40,538 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 40,000 axioms...
2026-10-28 23:04:40,732 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 50,000 axioms...
2026-10-28 23:04:40,898 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 60,000 axioms...
2026-10-28 23:04:41,051 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 70,000 axioms...
2026-10-28 23:04:41,199 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 80,000 axioms...
2026-10-28 23:04:41,329 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 90,000 axioms...
2026-10-28 23:04:41,490 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 100,000 axioms...
2026-10-28 23:04:41,666 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 110,000 axioms...
2026-10-28 23:04:41,844 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 120,000 axioms...
2026-10-28 23:04:41,984 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 130,000 axioms...
2026-10-28 23:04:42,117 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 140,000 axioms...
2026-10-28 23:04:42,241 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 150,000 axioms...
2026-10-28 23:04:42,423 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 160,000 axioms...
2026-10-28 23:04:42,585 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 170,000 axioms...
2026-10-28 23:04:42,766 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 180,000 axioms...
2026-10-28 23:04:42,962 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 190,000 axioms...
2026-10-28 23:04:43,274 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 200,000 axioms...
2026-10-28 23:04:43,486 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 210,000 axioms...
2026-10-28 23:04:43,695 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 220,000 axioms...
2026-10-28 23:04:43,819 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 230,000 axioms...
2026-10-28 23:04:43,980 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 240,000 axioms...
2026-10-28 23:04:44,107 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 250,000 axioms...
2026-10-28 23:04:44,233 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 260,000 axioms...
2026-10-28 23:04:44,360 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 270,000 axioms...
2026-10-28 23:04:44,485 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 280,000 axioms...
2026-10-28 23:04:44,607 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 290,000 axioms...
2026-10-28 23:04:44,749 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 300,000 axioms...
2026-10-28 23:04:44,895 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 310,000 axioms...
2026-10-28 23:04:45,037 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 320,000 axioms...
2026-10-28 23:04:45,186 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 330,000 axioms...
2026-10-28 23:04:45,354 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 340,000 axioms...
2026-10-28 23:04:45,539 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 350,000 axioms...
2026-10-28 23:04:45,741 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 360,000 axioms...
2026-10-28 23:04:45,918 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 370,000 axioms...
2026-10-28 23:04:46,056 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.AxiomDeserialiser - Deserialised 380,000 axioms...
2026-10-28 23:04:46,152 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 419691 reference set members read from sct2_sRefset_OWLExpressionSnapshot_US1000124_20260301.txt
2026-10-28 23:04:46,152 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading reference set members
2026-10-28 23:04:46,163 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 172 reference set members read from der2_cissccRefset_MRCMAttributeDomainSnapshot_US1000124_20260301.txt
2026-10-28 23:04:46,167 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Release files read. JVM total memory is approx 4,096 MB.
2026-10-28 23:04:46,353 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomyBuilder - Loaded release snapshot
2026-10-28 23:04:46,353 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomyBuilder - Time taken deserialising axioms 5.179s
2026-10-28 23:04:46,353 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomyBuilder - Loading delta
2026-10-28 23:04:46,355 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/sct2_Concept_Delta_INT_20261028.txt
2026-10-28 23:04:46,356 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/sct2_Relationship_Delta_INT_20261028.txt
2026-10-28 23:04:46,357 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/sct2_StatedRelationship_Delta_INT_20261028.txt
2026-10-28 23:04:46,358 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/der2_sRefset_OWLAxiomDelta_INT_20261028.txt
2026-10-28 23:04:46,359 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Unzipping file to /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/der2_ssRefset_ModuleDependencyDelta_INT_20261028.txt
2026-10-28 23:04:46,361 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Loading DELTA release files ReleaseFiles{conceptPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/sct2_Concept_Delta_INT_20261028.txt], descriptionPaths=[], textDefinitionPaths=[], relationshipPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/sct2_Relationship_Delta_INT_20261028.txt], concreteRelationshipPaths=[], statedRelationshipPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/sct2_StatedRelationship_Delta_INT_20261028.txt], identifierPaths=[], refsetPaths=[/var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/der2_sRefset_OWLAxiomDelta_INT_20261028.txt, /var/folders/pf/h86c6rxx53d2c5dbgz8020pw0000gp/T/temp-rf2-unzip12473797599194024322/der2_ssRefset_ModuleDependencyDelta_INT_20261028.txt]}
2026-10-28 23:04:46,362 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading concepts
2026-10-28 23:04:46,366 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 272 concepts read from sct2_Concept_Delta_INT_20261028.txt
2026-10-28 23:04:46,366 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - includedReferenceSetPathPatterns: [.*_sRefset_OWL.*, .*_cissccRefset_MRCMAttributeDomain.*]
2026-10-28 23:04:46,366 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - refset 'der2_sRefset_OWLAxiomDelta_INT_20261028.txt' matches pattern '.*_sRefset_OWL.*'
2026-10-28 23:04:46,366 [WARN ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 2 reference set filename patterns provided but only 1 file matches found. Patterns: [.*_sRefset_OWL.*, .*_cissccRefset_MRCMAttributeDomain.*], Matches: [der2_sRefset_OWLAxiomDelta_INT_20261028.txt]
2026-10-28 23:04:46,366 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading relationships
2026-10-28 23:04:46,366 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 0 relationships read from sct2_Relationship_Delta_INT_20261028.txt
2026-10-28 23:04:46,366 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading concrete relationships
2026-10-28 23:04:46,366 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading identifiers
2026-10-28 23:04:46,366 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading relationships
2026-10-28 23:04:46,367 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 0 relationships read from sct2_StatedRelationship_Delta_INT_20261028.txt
2026-10-28 23:04:46,367 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Reading reference set members
2026-10-28 23:04:46,373 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - 272 reference set members read from der2_sRefset_OWLAxiomDelta_INT_20261028.txt
2026-10-28 23:04:46,373 [INFO ] [main] org.ihtsdo.otf.snomedboot.ReleaseImporter - Release files read. JVM total memory is approx 4,096 MB.
2026-10-28 23:04:46,378 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomyBuilder - Loaded delta
2026-10-28 23:04:46,378 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomyBuilder - Time taken deserialising axioms 0s
2026-10-28 23:04:46,379 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomyBuilder - SnomedTaxonomy loaded in 40.648627042 seconds
2026-10-28 23:04:46,379 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomyBuilder - 386382 concepts loaded
2026-10-28 23:04:46,379 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomyBuilder - 0 active stated relationships loaded
2026-10-28 23:04:46,472 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomyBuilder - 387109 active axioms loaded
2026-10-28 23:04:46,472 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: Build existing taxonomy took 40.747 seconds
2026-10-28 23:04:46,472 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - Creating OwlOntology
2026-10-28 23:04:46,473 [INFO ] [main] org.snomed.otf.owltoolkit.taxonomy.SnomedTaxonomy - Using never grouped role list from MRCM reference set [1230370004, 733931002, 736476002, 736473005, 272741003, 774081006, 766939001, 774158006, 736518005, 1148793005, 1148969005, 733932009, 840560000, 1142141006, 860781008, 736475003, 836358009, 411116001, 1149367008, 1148967007, 736472000, 827081001, 1142139005, 733930001, 736474004, 320091000221107, 1142143009, 733933004, 1142140007, 1148965004, 1148968002, 763032000, 726542003, 733928003, 774159003, 738774007]
2026-10-28 23:04:47,572 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: Create OWL Ontology took 1.1 seconds
2026-10-28 23:04:47,572 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - Creating OwlReasoner
2026-10-28 23:04:47,583 [INFO ] [main] org.semanticweb.elk.config.ConfigurationFactory - Loading default configuration parameters for class org.semanticweb.elk.reasoner.config.ReasonerConfiguration
2026-10-28 23:04:47,865 [INFO ] [main] org.semanticweb.elk.reasoner.Reasoner - ELK reasoner was created
2026-10-28 23:04:47,865 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: Create reasoner took 0.293 seconds
2026-10-28 23:04:47,865 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - OwlReasoner inferring class hierarchy
2026-10-28 23:04:47,866 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Loading of Axioms started
2026-10-28 23:04:47,885 [INFO ] [main] org.semanticweb.elk.reasoner.stages.AbstractReasonerStage - Loading of Axioms using 12 workers
Loading of Axioms ...
Loading ...
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
11%
12%
13%
2026-10-28 23:04:48,266 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1142141006> <http://snomed.info/id/762706009>)
14%
15%
2026-10-28 23:04:48,314 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1142139005> <http://snomed.info/id/762706009>)
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
26%
27%
28%
29%
30%
31%
32%
33%
34%
35%
2026-10-28 23:04:48,616 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1142143009> <http://snomed.info/id/762706009>)
36%
37%
38%
39%
40%
41%
42%
2026-10-28 23:04:48,721 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1148793005> <http://snomed.info/id/762706009>)
43%
44%
45%
46%
2026-10-28 23:04:48,798 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1142136003> <http://snomed.info/id/762706009>)
47%
48%
49%
50%
51%
52%
53%
2026-10-28 23:04:48,902 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1142140007> <http://snomed.info/id/762706009>)
54%
2026-10-28 23:04:48,933 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1142138002> <http://snomed.info/id/762706009>)
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
71%
72%
73%
2026-10-28 23:04:49,285 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1142142004> <http://snomed.info/id/762706009>)
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
2026-10-28 23:04:49,504 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1142135004> <http://snomed.info/id/762706009>)
86%
87%
88%
89%
2026-10-28 23:04:49,552 [WARN ] [main] org.semanticweb.elk.reasoner.indexing.hierarchy.ChangeIndexingProcessor - [reasoner.indexing.axiomIgnored]ELK does not support SubDataPropertyOf. Axiom ignored:
SubDataPropertyOf(<http://snomed.info/id/1142137007> <http://snomed.info/id/762706009>)
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
... finished
... finished
2026-10-28 23:04:49,716 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Loading of Axioms took 1832 ms
2026-10-28 23:04:49,716 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Property Saturation Initialization started
2026-10-28 23:04:49,717 [INFO ] [main] org.semanticweb.elk.reasoner.stages.AbstractReasonerStage - Property Saturation Initialization using 12 workers
Property Saturation Initialization ...
... finished
2026-10-28 23:04:49,717 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Property Saturation Initialization took 0 ms
2026-10-28 23:04:49,717 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Reflexive Property Computation started
2026-10-28 23:04:49,720 [INFO ] [main] org.semanticweb.elk.reasoner.stages.AbstractReasonerStage - Reflexive Property Computation using 12 workers
Reflexive Property Computation ...
... finished
2026-10-28 23:04:49,724 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Reflexive Property Computation took 6 ms
2026-10-28 23:04:49,724 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Object Property Hierarchy and Composition Computation started
2026-10-28 23:04:49,726 [INFO ] [main] org.semanticweb.elk.reasoner.stages.AbstractReasonerStage - Object Property Hierarchy and Composition Computation using 12 workers
Object Property Hierarchy and Composition Computation ...
... finished
2026-10-28 23:04:49,727 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Object Property Hierarchy and Composition Computation took 3 ms
2026-10-28 23:04:49,727 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Context Initialization started
2026-10-28 23:04:49,728 [INFO ] [main] org.semanticweb.elk.reasoner.stages.AbstractReasonerStage - Context Initialization using 12 workers
Context Initialization ...
... finished
2026-10-28 23:04:49,729 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Context Initialization took 1 ms
2026-10-28 23:04:49,729 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Consistency Checking started
2026-10-28 23:04:49,731 [INFO ] [main] org.semanticweb.elk.reasoner.stages.AbstractReasonerStage - Consistency Checking using 12 workers
Consistency Checking ...
... finished
2026-10-28 23:04:49,734 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Consistency Checking took 5 ms
2026-10-28 23:04:49,734 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Class Taxonomy Computation started
2026-10-28 23:04:49,739 [INFO ] [main] org.semanticweb.elk.reasoner.stages.AbstractReasonerStage - Class Taxonomy Computation using 12 workers
Class Taxonomy Computation ...
4%
12%
14%
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
26%
27%
28%
29%
30%
31%
32%
33%
34%
35%
36%
37%
38%
39%
40%
41%
42%
43%
44%
45%
46%
47%
48%
49%
50%
51%
52%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
71%
72%
73%
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
... finished
2026-10-28 23:04:52,801 [INFO ] [main] org.semanticweb.elk.reasoner.stages.LoggingStageExecutor - Class Taxonomy Computation took 3067 ms
2026-10-28 23:04:52,802 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: Inference computation took 4.937 seconds
2026-10-28 23:04:52,802 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - Extract ReasonerTaxonomy
2026-10-28 23:04:52,815 [INFO ] [main] org.snomed.otf.owltoolkit.classification.ReasonerTaxonomyWalker - >>> SnomedTaxonomy extraction
2026-10-28 23:05:14,420 [INFO ] [main] org.snomed.otf.owltoolkit.classification.ReasonerTaxonomyWalker - <<< taxonomy extraction
2026-10-28 23:05:14,424 [INFO ] [main] org.semanticweb.elk.reasoner.Reasoner - ELK reasoner has shut down
2026-10-28 23:05:14,424 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: Extract ReasonerTaxonomy took 21.622 seconds
2026-10-28 23:05:14,424 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - Generate normal form
2026-10-28 23:05:15,197 [INFO ] [main] org.snomed.otf.owltoolkit.normalform.RelationshipNormalFormGenerator - Initialising node graphs for traversable properties [774081006, 733928003, 738774007, 733930001]
2026-10-28 23:05:15,199 [INFO ] [main] org.snomed.otf.owltoolkit.normalform.RelationshipNormalFormGenerator - >>> Relationship normal form generation
2026-10-28 23:05:27,854 [INFO ] [main] org.snomed.otf.owltoolkit.normalform.RelationshipNormalFormGenerator - <<< Relationship normal form generation [12.65 s]
2026-10-28 23:05:27,854 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: Generate normal form took 13.43 seconds
2026-10-28 23:05:27,855 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - Inactivating inferred relationships for new inactive concepts
2026-10-28 23:05:27,888 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - 3,476 relationship rows changed: 2,361 added, 0 updated, 1,115 redundant, 0 removed due to concept inactivation.
2026-10-28 23:05:27,888 [INFO ] [main] org.snomed.otf.owltoolkit.service.SnomedReasonerService - Writing results archive
2026-10-28 23:05:27,924 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: Write results to disk took 0.07 seconds
2026-10-28 23:05:27,924 [INFO ] [main] org.snomed.otf.owltoolkit.util.TimerUtil - Timer Classification: total took 82.202 seconds
Classification results written to /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/data/rf2_output/classification-results-2026-10-28_23-04-05.zip
2026-10-28 23:05:29,363 Classification complete → /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/data/rf2_output/classification-results-2026-10-28_23-04-05.zip
Classification results: /Users/aostrop1/Library/CloudStorage/OneDrive-JNJ/projects/aostropolets_git/Ariadne/data/rf2_output/classification-results-2026-10-28_23-04-05.zip
Parse results and resolve names¶
Extract the new inferred "Is a" relationships and resolve SNOMED concept names.
new_is_a, removed, equiv_df = parse_classification_results(results_zip)
print(f"New inferred 'Is a' relationships: {len(new_is_a)}")
print(f"Redundant relationships removed: {len(removed)}")
print(f"Equivalent concept rows: {len(equiv_df)}")
# Build lookup structures
synth_to_omop = dict(zip(
id_map["synthetic_sctid"].astype(str),
id_map["omop_concept_id"].astype(str),
))
synth_ids = set(id_map["synthetic_sctid"].astype(str))
# src_name_map: source ID → concept name (from id_map + attribute_gs override)
src_name_map = {}
for _, m in id_map.iterrows():
omop_str = str(m["omop_concept_id"])
if str(m.get("concept_name", "")):
src_name_map[omop_str] = str(m["concept_name"])
for _, gs_row in attribute_gs.iterrows():
sid = gs_row["source_concept_id"]
key = str(sid) if pd.notna(sid) else f"{gs_row['source_vocabulary_id']}:{gs_row['source_concept_code']}"
src_name_map[key] = gs_row["source_term"]
# Collect all real SCTIDs that appear as ELK results (parents or children)
real_sctids = set()
for _, row in new_is_a.iterrows():
src, dest = str(row["sourceId"]), str(row["destinationId"])
src_is_synth = src in synth_ids
dest_is_synth = dest in synth_ids
if src_is_synth and not dest_is_synth:
real_sctids.add(dest)
elif not src_is_synth and dest_is_synth:
real_sctids.add(src)
# Resolve SCTID → concept_name for display (keep SCTID as the ID)
sctid_name_map: dict[str, str] = {}
if real_sctids:
with psycopg.connect(conn_str) as conn:
with conn.cursor() as cur:
cur.execute(
f"""
SELECT concept_code, concept_name
FROM {schema}.concept
WHERE vocabulary_id = 'SNOMED'
AND concept_code = ANY(%s)
""",
(list(real_sctids),),
)
for code, name in cur.fetchall():
sctid_name_map[str(code)] = name
unmapped = real_sctids - set(sctid_name_map)
if unmapped:
print(f"{len(unmapped)} SCTIDs not found in CONCEPT table (will keep SCTID as fallback)")
# Build equivalence pairs from the ELK equivalences refset
# Group by mapTarget UUID — concepts sharing the same group are equivalent
from collections import defaultdict
equiv_groups: dict[str, list[str]] = defaultdict(list)
for _, row in equiv_df.iterrows():
sctid = str(row["referencedComponentId"])
group = str(row["mapTarget"])
equiv_groups[group].append(sctid)
# Generate "Maps to" pairs from equivalence groups (all pairwise within group)
equiv_pairs_set: set[tuple[str, str]] = set()
for group_members in equiv_groups.values():
# Only keep pairs where at least one member is a synthetic (delta) concept
synth_members = [m for m in group_members if m in synth_ids]
if not synth_members:
continue
for i, a in enumerate(group_members):
for b in group_members[i+1:]:
if a in synth_ids or b in synth_ids:
equiv_pairs_set.add((min(a, b), max(a, b)))
print(f"Equivalence groups: {len(equiv_groups)}, pairs involving delta concepts: {len(equiv_pairs_set)}")
# Build parents_df
rows = []
skipped_base_only = 0
for _, row in new_is_a.iterrows():
src, dest = str(row["sourceId"]), str(row["destinationId"])
src_is_synth = src in synth_ids
dest_is_synth = dest in synth_ids
if src_is_synth and dest_is_synth:
continue # inter-delta hierarchy handled via equivalences
if not src_is_synth and not dest_is_synth:
skipped_base_only += 1
continue
if src_is_synth and not dest_is_synth:
omop_id = synth_to_omop[src]
rows.append({
"source_concept_id": omop_id,
"source_term": src_name_map.get(omop_id, omop_id),
"relationship_id": "Is a",
"target_concept_code": dest,
"target_concept_name": sctid_name_map.get(dest, dest),
})
else:
omop_id = synth_to_omop[dest]
rows.append({
"source_concept_id": omop_id,
"source_term": src_name_map.get(omop_id, omop_id),
"relationship_id": "Subsumes",
"target_concept_code": src,
"target_concept_name": sctid_name_map.get(src, src),
})
# Add equivalent pairs as "Maps to"
for a, b in equiv_pairs_set:
a_is_synth = a in synth_ids
b_is_synth = b in synth_ids
if a_is_synth and b_is_synth:
# Both are delta concepts → use source IDs
omop_a = synth_to_omop[a]
omop_b = synth_to_omop[b]
rows.append({
"source_concept_id": omop_a,
"source_term": src_name_map.get(omop_a, omop_a),
"relationship_id": "Maps to",
"target_concept_code": omop_b,
"target_concept_name": src_name_map.get(omop_b, omop_b),
})
elif a_is_synth:
# a is delta, b is real SNOMED
omop_a = synth_to_omop[a]
rows.append({
"source_concept_id": omop_a,
"source_term": src_name_map.get(omop_a, omop_a),
"relationship_id": "Maps to",
"target_concept_code": b,
"target_concept_name": sctid_name_map.get(b, b),
})
else:
# b is delta, a is real SNOMED
omop_b = synth_to_omop[b]
rows.append({
"source_concept_id": omop_b,
"source_term": src_name_map.get(omop_b, omop_b),
"relationship_id": "Maps to",
"target_concept_code": a,
"target_concept_name": sctid_name_map.get(a, a),
})
parents_df = pd.DataFrame(rows, columns=[
"source_concept_id", "source_term", "relationship_id",
"target_concept_code", "target_concept_name",
])
parents_csv = project_root / "data" / "notebook_results" / "hierarchy_classification_parents.csv"
parents_df.to_csv(parents_csv, index=False)
n_maps_to = (parents_df["relationship_id"] == "Maps to").sum()
print(f"Saved {len(parents_df)} parent relationships → hierarchy_classification_parents.csv")
print(f" 'Is a' : {(parents_df['relationship_id']=='Is a').sum()}")
print(f" 'Subsumes' : {(parents_df['relationship_id']=='Subsumes').sum()}")
print(f" 'Maps to' : {n_maps_to}")
parents_df.head(20)
2026-10-28 23:05:29,435 EQUIVALENT CONCEPTS FOUND (378 rows)! This usually indicates a modelling error — two concepts have identical defining attributes. 2026-10-28 23:05:29,458 Classification results: 1516 new 'Is a' relationships, 1115 redundant removals, 378 equivalent concept rows.
New inferred 'Is a' relationships: 1516 Redundant relationships removed: 1115 Equivalent concept rows: 378 48 SCTIDs not found in CONCEPT table (will keep SCTID as fallback) Equivalence groups: 187, pairs involving delta concepts: 198 Saved 1691 parent relationships → hierarchy_classification_parents.csv 'Is a' : 502 'Subsumes' : 991 'Maps to' : 198
| source_concept_id | source_term | relationship_id | target_concept_code | target_concept_name | |
|---|---|---|---|---|---|
| 0 | 73555 | Transient arthropathy of the pelvic region and... | Is a | 66191007 | Transient arthropathy |
| 1 | 72404 | Joint stiffness | Subsumes | 202510005 | Multiple stiff joints |
| 2 | 31057 | Disorder of pharynx | Subsumes | 2129002 | Edema of pharynx |
| 3 | 27835 | Neoplasm of uncertain behavior of pituitary gland | Subsumes | 608817003 | Pituicytoma |
| 4 | 30770 | Chronic peptic ulcer with hemorrhage but witho... | Is a | 49232000 | Chronic peptic ulcer with hemorrhage |
| 5 | 30770 | Chronic peptic ulcer with hemorrhage but witho... | Is a | 128292002 | Chronic disease of cardiovascular system |
| 6 | 30770 | Chronic peptic ulcer with hemorrhage but witho... | Is a | 1142281007 | 1142281007 |
| 7 | 30770 | Chronic peptic ulcer with hemorrhage but witho... | Is a | 405539004 | Hemorrhage of blood vessel |
| 8 | 72745 | Open dislocation of elbow | Subsumes | 208806005 | Open traumatic subluxation of elbow |
| 9 | 24134 | Neck pain | Subsumes | 417347005 | Cervical somatic dysfunction |
| 10 | 72997 | Articular cartilage disorder of upper arm | Subsumes | 1344802008 | Osteochondropathy of radius |
| 11 | 72407 | Effusion of joint of shoulder region | Subsumes | 306981000119101 | Effusion of joint of right shoulder region |
| 12 | 72989 | Arthropathy associated with mycoses, of the an... | Is a | 201540009 | Arthropathy associated with mycoses, of the an... |
| 13 | 26069 | Postablative testicular hypofunction | Is a | 190552005 | Postablative testicular hypofunction |
| 14 | 72468 | Open dislocation of jaw | Is a | 208748005 | Open dislocation of jaw |
| 15 | 72421 | Solitary bone cyst | Is a | 203465002 | Bone cyst |
| 16 | 72566 | Primary malignant neoplasm of pelvic bones, sa... | Subsumes | 93842002 | Primary malignant neoplasm of ischium |
| 17 | 73553 | Arthropathy | Subsumes | 715470008 | Brachydactylous dwarfism Mseleni type |
| 18 | 72404 | Joint stiffness | Subsumes | 321871000119109 | Stiffness of right wrist |
| 19 | 73553 | Arthropathy | Subsumes | 3723001 | Arthritis |
Evaluation: inferred parents vs parent gold standard¶
Compare the ELK-inferred "Is a" parents against the actual direct "Is a" relationships
from the parent gold standard (hierarchy_snomed_gs.csv)
# ── Full per-concept evaluation file ──────────────────────────────────────
def _join(df: pd.DataFrame, sid: str, column: str) -> str:
values = df.loc[df["source_concept_id"] == sid, column].astype(str).drop_duplicates()
return ";".join(sorted(values))
source_term_map = {str(r["source_concept_id"]): r["source_term"] for _, r in attribute_gs.iterrows()}
eval_rows = []
for sid in sorted(source_ids):
gs_codes = {code for s, code in actual_set if s == sid}
pred_codes = {code for s, code in predicted_is_a if s == sid}
matched_codes = gs_codes & pred_codes
is_a_itself = sid in is_a_to_itself_concepts
if sid in maps_to_correct_concepts:
outcome = "maps_to_correct"
elif sid in maps_to_incorrect_concepts:
outcome = "maps_to_incorrect"
elif is_a_itself:
outcome = "is_a_to_itself"
elif matched_codes:
outcome = "is_a_correct_parent"
elif pred_codes:
outcome = "is_a_incorrect"
elif sid in subsumes_concepts:
outcome = "subsumes_only"
else:
outcome = "no_relationship"
eval_rows.append({
"source_concept_id": sid,
"source_term": source_term_map.get(sid, ""),
"source_concept_code": source_code_map.get(sid, ""),
"gs_parent_codes": ";".join(sorted(gs_codes)),
"gs_parent_names": _join(actual_gs, sid, "target_concept_name"),
"predicted_parent_codes": ";".join(sorted(pred_codes)),
"predicted_parent_names": _join(is_a_df, sid, "target_concept_name"),
"matched_parent_codes": ";".join(sorted(matched_codes)),
"maps_to_codes": _join(maps_to_df, sid, "target_concept_code"),
"maps_to_names": _join(maps_to_df, sid, "target_concept_name"),
"subsumes_codes": _join(subsumes_df, sid, "target_concept_code"),
"subsumes_names": _join(subsumes_df, sid, "target_concept_name"),
"n_gs_parents": len(gs_codes),
"n_predicted_parents": len(pred_codes),
"n_matched_parents": len(matched_codes),
"is_maps_to": sid in maps_to_concepts,
"is_maps_to_correct": sid in maps_to_correct_concepts,
"is_a_to_itself": is_a_itself,
"is_a_correct_parent": bool(matched_codes),
"has_is_a": bool(pred_codes),
"has_subsumes": sid in subsumes_concepts,
"outcome": outcome,
})
classification_eval = pd.DataFrame(eval_rows)
classification_eval.to_csv(project_root / "data" / "notebook_results" / "hierarchy_classification_evaluation.csv", index=False)
print(f"\nSaved {len(classification_eval)} concept rows to hierarchy_classification_evaluation.csv")
print(classification_eval["outcome"].value_counts().to_string())
classification_eval.head()
Saved 275 concept rows to hierarchy_classification_evaluation.csv outcome maps_to_correct 171 is_a_correct_parent 31 is_a_to_itself 28 is_a_incorrect 21 maps_to_incorrect 20 no_relationship 3 subsumes_only 1
| source_concept_id | source_term | source_concept_code | gs_parent_codes | gs_parent_names | predicted_parent_codes | predicted_parent_names | matched_parent_codes | maps_to_codes | maps_to_names | ... | n_gs_parents | n_predicted_parents | n_matched_parents | is_maps_to | is_maps_to_correct | is_a_to_itself | is_a_correct_parent | has_is_a | has_subsumes | outcome | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 193782 | End-stage renal disease | 46177005 | 153611000112104;90688005 | Chronic renal failure;End stage organ failure | 128290005;236423003 | Chronic disease of genitourinary system;Renal ... | 709044004 | Chronic kidney disease | ... | 2 | 2 | 0 | True | False | False | False | True | True | maps_to_incorrect | |
| 1 | 22274 | Neoplasm of uncertain behavior of larynx | 94899001 | 126692004;94962000;95158003 | Neoplasm of larynx;Neoplasm of uncertain behav... | 248595008;94899001 | Neoplasm of uncertain behavior of larynx;Sputu... | ... | 3 | 2 | 0 | False | False | True | False | True | False | is_a_to_itself | |||
| 2 | 22288 | Hereditary elliptocytosis | 191169008 | 234409003;276654001;363070008;38911009;42601008 | Congenital hemolytic anemia;Congenital malform... | 276654001;323666000;363070008;414394009;42601008 | Anemia due to intrinsic red cell abnormality;C... | 276654001;363070008;42601008 | ... | 5 | 5 | 3 | False | False | False | True | True | True | is_a_correct_parent | ||
| 3 | 22350 | Edema of larynx | 51599000 | 118654009;60600009 | Disorder characterized by edema;Disorder of th... | 51599000 | Edema of larynx | 51599000 | Edema of larynx | ... | 2 | 1 | 0 | True | True | True | False | True | False | maps_to_correct | |
| 4 | 22492 | Foreign body in pharynx | 25479004 | 128518001;33334006;75860007 | Disorder of pharynx;Foreign body in digestive ... | 25479004 | Foreign body in pharynx | 25479004 | Foreign body in pharynx | ... | 3 | 1 | 0 | True | True | True | False | True | False | maps_to_correct |
5 rows × 22 columns