A Package Skeleton for Patient-Level Prediction Studies

A skeleton package, to be used as a starting point when implementing patient-level prediction studies.

Vignette: Using the package skeleton for patient-level prediction studies

For information on how to generate shiny apps using the R skeleton package see: Creating shiny app for patient-level prediction studies. To combine multiple skeleton package results into a single shiny app see populate multiple shiny app R code code

For information on adding custom covariates based on ATLAS cohorts see: Adding custom covariates using ATLAS cohorts.

Suggested Requirements

Instructions To Build Package

  • Open the package project (file ending with ‘.Rproj’) by double clicking it
  • Install any missing dependancies for the package by running:
source('./extras/packageDeps.R')
  • Build the package by clicking the R studio ‘Install and Restart’ button in the build tab (top right)

Instructions To Run Study

  • Execute the study by running the code in (extras/CodeToRun.R) :
  library(SkeletonPredictionStudy)
  # USER INPUTS
#=======================
# The folder where the study intermediate and result files will be written:
outputFolder

The ‘createCohorts’ option will create the target and outcome cohorts into cohortDatabaseSchema.cohortTable if set to T. The ‘runAnalyses’ option will create/extract the data for each prediction problem setting (each Analysis), develop a prediction model, internally validate it if set to T. The results of each Analysis are saved in the ‘outputFolder’ directory under the subdirectories ‘Analysis_1’ to ‘Analysis_N’, where N is the total analyses specified. After running execute with ‘runAnalyses set to T, a ’Validation’ subdirectory will be created in the ‘outputFolder’ directory where you can add the external validation results to make them viewable in the shiny app or journal document that can be automatically generated.

  • You can then easily transport all the trained models into a network validation study package by running:
  
  execute(connectionDetails = connectionDetails,
        cdmDatabaseSchema = cdmDatabaseSchema,
        cdmDatabaseName = cdmDatabaseName,
        cohortDatabaseSchema = cohortDatabaseSchema,
        cohortTable = cohortTable,
        outputFolder = outputFolder,
        createValidationPackage = T)
  • To pick specific models (e.g., models from Analysis 1 and 3) to export to a validation study run:
  
  execute(connectionDetails = connectionDetails,
        cdmDatabaseSchema = cdmDatabaseSchema,
        cdmDatabaseName = cdmDatabaseName,
        cohortDatabaseSchema = cohortDatabaseSchema,
        cohortTable = cohortTable,
        outputFolder = outputFolder,
        createValidationPackage = T, 
        analysesToValidate = c(1,3))

This will create a new subdirectory in ‘outputFolder’ that has the name Validation. For example, if your prediction study package was named ‘bestPredictionEver’ and you run the execute with ‘createValidationPackage’ set to T with ‘outputFolder’= ‘C:/myResults’, then you will find a new R package at the directory: ‘C:/myResults/bestPredictionEverValidation’. This package can be executed similarly but will validate the developed model/s rather than develop new model/s. If you set the validation package outputFolder to the Validation directory of the prediction package results (e.g., ‘C:/myResults/Validation’), then the results will be saved in a way that can be viewed by shiny.

  • To create the shiny app and view the results, run the following after successfully developing the models:
  
execute(connectionDetails = connectionDetails,
        cdmDatabaseSchema = cdmDatabaseSchema,
        cdmDatabaseName = cdmDatabaseName,
        cohortDatabaseSchema = cohortDatabaseSchema,
        cohortTable = cohortTable,
        outputFolder = outputFolder,
        createShiny = T,
        minCellCount= 5)
PatientLevelPrediction::viewMultiplePlp(outputFolder)

If you saved the validation results into the validation folder in the directory you called ‘outputFolder’ in the structure: ‘/Validation//Analysis_N’ then shiny and the journal document creator will automatically include any validation results. The validation package will automatically save the validation results in this structure if you set the outputFolder for the validation results to: ‘/Validation’.

  • To create the journal document for the Analysis 1:
  
execute(connectionDetails = connectionDetails,
        cdmDatabaseSchema = cdmDatabaseSchema,
        cdmDatabaseName = cdmDatabaseName,
        cohortDatabaseSchema = cohortDatabaseSchema,
        cohortTable = cohortTable,
        outputFolder = outputFolder,
        createJournalDocument = F,
        analysisIdDocument = 1
        minCellCount= 5)

Instructions To Share Package

  • Share the package by adding it to the ‘ohdsi-studies’ github repo and get people to install by:
  # get the latest PatientLevelPrediction
  install.packages("devtools")
  devtools::install_github("OHDSI/PatientLevelPrediction")
  # check the package
  PatientLevelPrediction::checkPlpInstallation()
  
  # install the network package
  devtools::install_github("ohdsi-studies/SkeletonPredictionStudy")

Development status

Under development. Do not use