AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance


JSON Export

{
  "updated": "2020-03-23T00:00:00+00:00", 
  "id": "343", 
  "revision": 1, 
  "created": "2020-05-12T13:53:48.641573+00:00", 
  "metadata": {
    "description": "The ever-growing availability of computing power and sustained development of advanced computational methods have contributed much to recent scientific progress.\r\nThese developments present new challenges driven by the sheer amount of calculations and data to manage. Next-generation exascale supercomputers will harden these challenges, such that automated and scalable solutions become crucial. In recent years, we have been developing AiiDA (http://www.aiida.net), a robust open-source high-throughput infrastructure addressing the challenges arising from the needs of automated workflow management and data provenance recording. Here, we introduce developments and capabilities required to reach sustained performance, with AiiDA supporting throughputs of tens of thousands processes/hour, while automatically preserving and storing the full data provenance in a relational database making it queryable and traversable, thus enabling high-performance data analytics. AiiDA's workflow language provides advanced automation, error handling features and a flexible plugin model to allow interfacing with any simulation software. The associated plugin registry enables seamless sharing of extensions, empowering a vibrant user community dedicated to making simulations more robust, user-friendly and reproducible.\r\n\r\nThis archive record contains the data to reproduce the figures on engine performance in the section \"Event versus polling-based engine\" of the paper entitled \"AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance\". It also includes instructions to reproduce the actual data from scratch using AiiDA v1.1.1 and AiiDA v0.12.5.", 
    "status": "published", 
    "title": "AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance", 
    "is_last": true, 
    "license": "Creative Commons Attribution 4.0 International", 
    "publication_date": "Mar 23, 2020, 00:00:00", 
    "id": "343", 
    "_oai": {
      "id": "oai:materialscloud.org:343"
    }, 
    "version": 1, 
    "keywords": [
      "reproducibility", 
      "workflows", 
      "high-throughput", 
      "automation", 
      "MARVEL", 
      "MaX", 
      "SNSF", 
      "PASC", 
      "PRACE", 
      "ERC", 
      "Swissuniversities", 
      "MarketPlace", 
      "Intersect", 
      "NFFA", 
      "EMMC"
    ], 
    "conceptrecid": "342", 
    "edited_by": 98, 
    "_files": [
      {
        "description": "Readme file in markdown format with full description of contents of the `export_archive.aiida` and `supplementary.tar.gz` files, as well as instructions on how to reproduce the data of the paper.", 
        "checksum": "md5:6371a774bfa2cc9e3b33557897a96c07", 
        "key": "README.md", 
        "size": 4077
      }, 
      {
        "description": "AiiDA export archive containing the actual data used to produce the figures in the referenced publication.", 
        "checksum": "md5:33de3d6fe9647d3b3fa0974a262db706", 
        "key": "export_archive.aiida", 
        "size": 6224629
      }, 
      {
        "description": "Archive containing the necessary scripts to completely reproduce the data contained within the `export_archive.aiida` AiiDA export archive and subsequently, analyse and plot the results.", 
        "checksum": "md5:6ddab17a17f8f0a1d61feff134580070", 
        "key": "supplementary.tar.gz", 
        "size": 13411
      }
    ], 
    "mcid": "2020.0027/v1", 
    "doi": "10.24435/materialscloud:2020.0027/v1", 
    "contributors": [
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Sebastiaan P.", 
        "familyname": "Huber", 
        "email": "mail@sphuber.net"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Spyros", 
        "familyname": "Zoupanos"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Martin", 
        "familyname": "Uhrin"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Laboratory of Molecular Simulation (LSMO), Institut des Sciences et Ing\u00e9nierie Chimiques, E\u0301cole Polytechnique F\u00e9d\u00e9rale de Lausanne (EPFL), Rue de l\u2019Industrie 17, Sion, CH-1951 Valais, Switzerland"
        ], 
        "givennames": "Leopold", 
        "familyname": "Talirz"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Leonid", 
        "familyname": "Kahle"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Rico", 
        "familyname": "H\u00e4uselmann"
      }, 
      {
        "affiliations": [
          "Microsoft Station Q, University of California, Santa Barbara, California 93106-6105, USA"
        ], 
        "givennames": "Dominik", 
        "familyname": "Gresch"
      }, 
      {
        "affiliations": [
          "Institut f\u00fcr Physikalische Chemie, University of Z\u00fcrich, Switzerland"
        ], 
        "givennames": "Tiziano", 
        "familyname": "M\u00fcller"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Laboratory of Molecular Simulation (LSMO), Institut des Sciences et Ing\u00e9nierie Chimiques, E\u0301cole Polytechnique F\u00e9d\u00e9rale de Lausanne (EPFL), Rue de l\u2019Industrie 17, Sion, CH-1951 Valais, Switzerland"
        ], 
        "givennames": "Aliaksandr V.", 
        "familyname": "Yakutovich"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Casper W.", 
        "familyname": "Andersen"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Francisco F.", 
        "familyname": "Ramirez"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Carl S.", 
        "familyname": "Adorf"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Fernando", 
        "familyname": "Gargiulo"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Snehal", 
        "familyname": "Kumbhar"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Elsa", 
        "familyname": "Passaro"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Conrad", 
        "familyname": "Johnston"
      }, 
      {
        "affiliations": [
          "Vilnius University Institute of Biotechnology, Saul\u0117tekio al. 7, LT-10257 Vilnius, Lithuania"
        ], 
        "givennames": "Andrius", 
        "familyname": "Merkys"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Andrea", 
        "familyname": "Cepellotti"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Nicolas", 
        "familyname": "Mounet"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Nicola", 
        "familyname": "Marzari"
      }, 
      {
        "affiliations": [
          "John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts 02138, USA", 
          "Robert Bosch LLC, Research and Technology Center North America, 255 Main St, Cambridge, Massachusetts 02142, USA"
        ], 
        "givennames": "Boris", 
        "familyname": "Kozinsky"
      }, 
      {
        "affiliations": [
          "National Centre for Computational Design and Discovery of Novel Materials (MARVEL), \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland", 
          "Theory and Simulation of Materials (THEOS), Facult\u00e9 des Sciences et Techniques de l'Ing\u00e9nieur, \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, CH-1015 Lausanne, Switzerland"
        ], 
        "givennames": "Giovanni", 
        "familyname": "Pizzi"
      }
    ], 
    "license_addendum": "", 
    "references": [
      {
        "comment": "", 
        "doi": "", 
        "type": "Journal reference", 
        "citation": "S. P. Huber, S. Zoupanos, M. Uhrin, L. Talirz, L. Kahle, R. H\u00e4uselmann, D. Gresch, T. M\u00fcller, A. V. Yakutovich, C. W. Andersen, F. F. Ramirez, C. S. Adorf, F. Gargiulo, S. Kumbhar, E. Passaro, C. Johnston, A. Merkys, A. Cepellotti, N. Mounet, N. Marzari, B. Kozinsky, G. Pizzi, (2020) submitted", 
        "url": ""
      }
    ], 
    "owner": 89
  }
}