TravisCI build status

pyEHR

pyEHR is a a scalable clinical data access layer intended for building data management systems for secondary use of structured heterogeneous biomedical and clinical data.

Documentation

API Documentation can be found here

Installation and setup

Download pyEHR and unzip it/clone it in a directory [path_to_pyEHR_dir]

Download basex from: http://basex.org/

In a shell run:

{path_to_basex_dir}/bin/basexhttp

Install Java 8 and export its path:

export JAVA_HOME={path_to_java8}

Install python dependencies (for example with pip:

sudo pip install --pre pybasex
sudo pip install voluptuous
sudo pip install bottle
sudo pip install httplib2

Data Management back-end

pyEHR is compatible with multiple back-end engines (right now MongoDB and Elasticsearch are the ones supported) so you can choose which one to install to run with the pyEHR environment.

Elasticsearch

Install Elasticsearch 1.5 or lower. run:

{path_to_elasticsearch}/bin/elasticsearch 

python module for elasticsearch:

sudo pip install -Iv elasticsearch==1.5.0

export environment variable with driver configuration:

export SERVICE_CONFIG_FILE={path_to_config_file}/config_elasticsearch.conf

Mongodb

Install and run MongoDB. python module for mongodb:

sudo pip install pymongo

export environment variable with driver configuration:

export SERVICE_CONFIG_FILE={path_to_config_file}/config_mongodb.conf

pyEHR environment

export path to pybasex and pyEHR:

export PYTHONPATH=${PYTHONPATH}:{path_to_pyEHR_dir}:{path_to_pybasex_dir}

install pyEHR:

sudo python {path_to_pyEHR_dir}/setup.py install

pyEHR is now ready to be used

USAGE

To create a dataset:

python  {path_to_pyEHR_dir}/test/misc/test_query_performance/datasets_builder.py --structures_config_file {path_to_structures_config_file}/structures_config_file --datasets_config_file {path_to_datasets_config_file}/datasets_config_file --archetypes_dir {path_to_archetypes_dir} --structures_out_file {path_to_structures_out_file}/structures_out_file --datasets_out_file {path_to_datasets_out_file}/datasets_out_file

To load a dataset:

python {path_to_pyEHR_dir}/test/misc/test_query_performance/datasets_loader.py
--datasets_file {path_to_dataset}/dataset.json --pyEHR_config {path_to_config_file}/config_elasticsearch.conf --parallel_processes 8

To query loading the queries from a set of queries:

python {path_to_pyEHR_dir}/test/misc/test_query_performance/run_queries.py --queries_file 
{path_to_queries_file}/queriesfile.json --query_processes 16 --log_file {path_to_log_file}/logfile.log 
--pyEHR_config {path_to_config_file}/config_elasticsearch.conf  --results_file {path_to_results_file}/resultsfile

FILE EXAMPLES

Example of config file for (SERVICE_CONFIG_FILE environment variable):

• [config/services.elasticsearch.conf] first
• [config/services.mongodb.conf] second

Example of queries file:

• [test/misc/test_query_performance/data/conf/queries_conf.json] third

Example of structures_config_file:

• [test/misc/test_query_performance/data/conf/structures_conf.json] fourth

Example of datasets_config_file in

• [test/misc/test_query_performance/data/conf/patients_dataset_conf.json] fifth

Example of archetype dir (with archetypes):

• [models/json] [sixth]

Dataset of 100 records in:

• https://github.com/crs4/pyEHR/tree/develop/test/misc/test_query_performance/data/datasets

Dataset of 10M records in:

• ftp://ftp.crs4.it/surfer/public/pyEHR_TESTING_DATA/DATASET_CONSTANT_NUMBER_OF_RECORDS/

10 Datasets of 1M records in:

• ftp://ftp.crs4.it/surfer/public/pyEHR_TESTING_DATA/DATASET_CONSTANT_NUMBER_OF_RECORDS/

TESTS

queries:

python test/pyEHR/ehr/services/dbmanager/querymanager/test_querymanager.py

version manager:

python test/pyEHR/ehr/services/dbmanager/dbservices/test_version_manager.py

elasticsearch:

python test/pyEHR/ehr/services/dbmanager/drivers/test_elasticsearch.py

mongodb:

python test/pyEHR/ehr/services/dbmanager/drivers/test_mongo.py

dbservices:

python test/pyEHR/ehr/services/dbmanager/dbservices/test_dbservices.py

REST services: from ./services/

python dbservice.py --config ../config/services.elasticsearch.conf
python test/services/test_dbservice.py

[sixth]: <https://github.com/crs4/pyEHR/tree/develop/models/json