The EOSC-hub project has ended. This space is READ ONLY

Principal Investigators: Zhiming Zhao (University of Amsterdam, z.zhao@uva.nl)

Shepherds: Andrea Manzi (EGI Foundation, andrea.manzi@egi.eu)

Entry in the community requirement database: ENVRI-FAIR

About the pilot

Description of supported work

The project goal is to deploy a DevOps environment, with necessary capacity of Cloud Infrastructures and services for testing ENVRI-FAIR development. The project aims to automate the testing/integration of the FAIR data services developed by the teams in ENVRI-FAIR.

Objectives

  • Automated Cloud execution for data workflow : Demonstrate it in the VREs or in ENVRI RIs (e.g., LifeWatch or others). It will help the ENVRI community to learn the EOSC services, and build practices for the other similar use cases
  • Continuously testing and integration for ENVRI services: get familiar with the DevOps/Agile methodologies for software development, testing and operation
  • Notebook based environment for FAIR data access and processing: : 1) provide the Jupyter service to  users, with examples to access data sets and models, 2) users can perform customised experiments using the notebook services, access data, store the data, publish and share the results with the others.

Team

ParticipantRoleName and Surname
ENVRI-FAIRPIZhiming Zhao 
EGI FoundationShepherdAndrea Manzi
ENVRI-FAIRDevelopersSpiros Koulouzis, Yuandou Wang,



Technical Plan

The full technical plan can be found here: 

Work planned for Q1

  1. Test services/tools developed in ENVRIplus in the test bed, (e.g., DRIP for OpenStack services, particle formation for OneData/Jupyter notebook, etc.)
  2. Get familiar with the services and test bed
  3. Test how to request resources based specific geo-locations
  4. Communicate with different sub domains/RIs to collect potential experiments on the test bed
  5. Make plan for the experiments

Work planned for Q2

  1. Develop demonstrator for the automated VM planning, provisioning and service deployment. Extended it to a scientific workflow;
  2. Set up the DevOps pipeline, and test CI/CD for data management services (e.g., knowledge base content update, and experiments collected from Q1)
  3. Perform data experiments in the Jupyter hub, and perform some benchmark studies on the performance

Work planned for Q3

  1. Develop demonstrator for a scientific workflow (e.g., from lifewatch or other sub-domains/RIs)
  2. Demonstrate the DevOps pipeline of Jelastic for data management services
  3. Demonstrate a number of Jupyter notebook examples

Work planned for Q4

  1. Exploitation of the results to the ENVRI-FAIR communities
  2. Support ENVRI-FAIR community to take over the results and further develop them in their own context
  3. Support ENVRI-FAIR community to prepare for the further EOSC integration

EOSC services and providers

Providers

CESGA:

TypeResourcesResource StatusService status
Jelastic Installation
  • 1 VM 8 CPUs , 24 GB RAM , 1 TB storage via NFS
  • 1 VM 12 CPUs, 24 GB RAM, 1.5 TB storage via NFS
Provisioned

Provisioned

https://jca.j.fedcloud.eu/

https://app.j.fedcloud.eu/

VM for Automated Cloud execution for data workflow

  • 1 VM 12 CPUs, 16 GB RAM, 1.5 TB storage via NFS
ProvisionedOK

VM and Storage for Oneprovider installation

  • 1 VM 8 CPUs, 32 GB RAM, 50 GB local storage
  • 10 TB storage via NFS
Provisioned

https://oneprovider-envri.datahub.egi.eu/


INFN-CATANIA:

TypeResourcesResource StatusService status

 EGI Notebooks instance

  • 4 VMs 8 CPUs, 16 GB RAM , 120GB local storage
Provisioned

Provisioned

https://envri-notebooks.fedcloud-tf.fedcloud.eu/

access to DataHub to be defined

VM for Automated Cloud execution for data workflow

  • 4 VMs 4 CPUs, 8GB RAM, 100 GB local storage
ProvisionedOK


Services

  • EGI Cloud Compute
  • EGI Notebooks
  • Jelastic
  • EGI DataHub



  • No labels

EOSC-hub receives funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No. 777536.