Result Set Export Manager

EXPERIMENTAL - this feature is still in design stage and it is not reccomended that you implement this for your package at this stage. Utility for simplifying export of results to files from sql queries

Note that this utility is not strictly thread safe though seperate processes can export separate tables without issue. When exporting a the same table across multiple threads primary key checks may create issues.

Public fields

exportDir

direcotry path to export files to Init

Methods

Public methods

• ResultExportManager$new()

• ResultExportManager$getTableSpec()

• ResultExportManager$getMinColValues()

• ResultExportManager$checkRowTypes()

• ResultExportManager$listTables()

• ResultExportManager$checkPrimaryKeys()

• ResultExportManager$exportDataFrame()

• ResultExportManager$exportQuery()

• ResultExportManager$getManifestList()

• ResultExportManager$writeManifest()

• ResultExportManager$clone()

---

Method new()

Create a class for exporting results from a study in a standard, consistend manner

Usage

ResultExportManager$new(
  tableSpecification,
  exportDir,
  minCellCount = getOption("ohdsi.minCellCount", default = 5),
  databaseId = NULL
)

Arguments

tableSpecification

Table specification data.frame

exportDir

Directory files are being exported to

minCellCount

Minimum cell count - reccomended that you set with options("ohdsi.minCellCount" = count) in all R projects. Default is 5

databaseId

database identifier - required when exporting according to many specs get table spec

---

Method getTableSpec()

Get specification of table

Usage

ResultExportManager$getTableSpec(exportTableName)

Arguments

exportTableName

table name Get min col values

---

Method getMinColValues()

Columns to convert to minimum for a given table name

Usage

ResultExportManager$getMinColValues(rows, exportTableName)

Arguments

rows

data.frame of rows

exportTableName

stering table name - must be defined in spec Check row types

---

Method checkRowTypes()

Check types of rows before exporting

Usage

ResultExportManager$checkRowTypes(rows, exportTableName)

Arguments

rows

data.frame of rows to export

exportTableName

table name List tables

---

Method listTables()

list all tables in schema Check primary keys of exported data

Usage

ResultExportManager$listTables()

---

Method checkPrimaryKeys()

Checks to see if the rows conform to the valid primary keys If the same table has already been checked in the life of this object set "invalidateCache" to TRUE as the keys will be cached in a temporary file on disk.

Usage

ResultExportManager$checkPrimaryKeys(
  rows,
  exportTableName,
  invalidateCache = FALSE
)

Arguments

rows

data.frame to export

exportTableName

Table name (must be in spec)

invalidateCache

logical - if starting a fresh export use this to delete cache of primary keys Export data frame

---

Method exportDataFrame()

This method is intended for use where exporting a data.frame and not a query from a rdbms table For example, if you perform a transformation in R this method will check primary keys, min cell counts and data types before writing the file to according to the table spec

Usage

ResultExportManager$exportDataFrame(rows, exportTableName, append = FALSE)

Arguments

rows

Rows to export

exportTableName

Table name

append

logical - if true will append the result to a file, otherwise the file will be overwritten Export Data table with sql query

---

Method exportQuery()

Writes files in batch to stop overflowing system memory Checks primary keys on write Checks minimum cell count

Usage

ResultExportManager$exportQuery(
  connection,
  sql,
  exportTableName,
  transformFunction = NULL,
  transformFunctionArgs = list(),
  append = FALSE,
  ...
)

Arguments

connection

DatabaseConnector connection instance

sql

OHDSI sql string to export tables

exportTableName

Name of table to export (in snake_case format)

transformFunction

(optional) transformation of the data set callback. must take two paramters - rows and pos

   Following this transformation callback, results will be verified against data model,
   Primary keys will be checked and minCellValue rules will be enforced

transformFunctionArgs

arguments to be passed to the transformation function

append

Logical add results to existing file, if FALSE (default) creates a new file and removes primary key validation cache

...

extra parameters passed to sql get manifest list

---

Method getManifestList()

Create a meta data set for each collection of result files with sha256 has for all files

Usage

ResultExportManager$getManifestList(
  packageName = NULL,
  packageVersion = NULL,
  migrationsPath = NULL,
  migrationRegexp = .defaultMigrationRegexp
)

Arguments

packageName

if an R analysis package, specify the name

packageVersion

if an analysis package, specify the version

migrationsPath

path to sql migrations (use top level folder (e.g. sql/sql_server/migrations)

migrationRegexp

(optional) regular expression to search for sql files. It is not reccomended to change the default. Write manifest

---

Method writeManifest()

Write manifest json

Usage

ResultExportManager$writeManifest(...)

Arguments

...

@seealso getManifestList

---

Method clone()

The objects of this class are cloneable with this method.

Usage

ResultExportManager$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.