This folder contains the raw data used for the section 'Event versus polling-based engine' of the paper:

AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance

The test consists of running 400 instances of a WorkChain that runs a CalcJob and a calcfunction that each execute a simple sum of two integers. The analysis consists of looking at the submission and completion rates of those processes after they have finished. The procedure essentially comes down to three steps:

1. Launch the workchains
2. Analyse the results for that batch of workchains
3. Plot the results after having done 1 and 2 for all environments of interest

The test has been performed for the 'new' engine, using AiiDA v1.1.1, and for the 'legacy' engine using a modified version of AiiDA v0.12.4. To reperform the tests, one would have to first install these two AiiDA versions in separate virtual environments and setup a profile. After the profile has been setup, the localhost computer should be configured followed by the add code that is to be used to run the ArithmeticAddCalculation. The add.sh bash script included in this folder can be used for that purpose.

Rerunning the analysis from existing data by importing export archive

To simply rerun the analysis on the existing data, simply import the included export archive in an AiiDA profile and rerun the included analysis script:

cd "${FILEPATH_ENGINE_PERFORMANCE}"
verdi import export_archive.aiida

python analysis.py engine_baseline
python analysis.py engine_constrained
python analysis.py engine_legacy

python plot_engine_performance.py

Reproducing the data manually

If you want to reproduce the data yourself, you will have to create two separate virtual environments with the two versions of AiiDA installed.

Installation instructions of AiiDA

FILEPATH_ENGINE_PERFORMANCE=''  # The absolute filepath to the folder containing this README

AiiDA v1.1.1

For the new version there are now specific installation instructions so from pip should suffice

pip install `aiida-core==1.1.1`

AiiDA v0.12.4

To install the exact version used for AiiDA v0.12.4 one should check out the corresponding tag from a local clone of the repository and apply the patch file included in this folder:

git clone git@github.com:aiidateam/aiida-core
cd aiida-core
git checkout v0.12.4
git apply "$FILEPATH_ENGINE_PERFORMANCE/legacy/patch.diff"
pip install -e .
reentry scan

Creating a profile and configuring the add code on localhost

Note that the exact commands for setting up the localhost as a computer and the add code depend on the version of AiiDA for which we refer to the documentation. For the add code, the add.sh bash script included in this folder can be used. In principle it boils down to the following commands:

verdi quicksetup   # Set up a profile
verdi computer setup localhost  # Set up `localhost` as a computer
verdi computer configure local localhost  # Configure the `localhost` computer
verdi code setup add  # Set up the `add` code on `localhost`

Rerunning the workchains

This step assumes that you have a fully installed and configured environment for both AiiDA versions.

Environment with AiiDA v1.1.1

cd "${FILEPATH_ENGINE_PERFORMANCE}"
export PYTHONPATH="$PYTHONPATH:$(pwd)/new"

verdi computer configure local localhost --safe-interval=0
verdi daemon start 12
python new/launch.py add@localhost engine_baseline -N 400

verdi daemon stop

verdi computer configure local localhost --safe-interval=5
verdi daemon start 4
python new/launch.py add@localhost engine_constrained -N 400

python analysis.py engine_baseline
python analysis.py engine_constrained

Environment with AiiDA v0.12.5

cd "${FILEPATH_ENGINE_PERFORMANCE}"
export PYTHONPATH="$PYTHONPATH:$(pwd)/legacy"

verdi daemon start

python new/launch.py add@localhost engine_legacy -N 400

python analysis.py engine_legacy