2020-01-21 Baseline Expectations Meeting

Date

21 Jan 2020 at 15:30 CET

Attendees

• Nicole Harris
• Casper Dreef
• Mario Reale 
• Tom Barton
• Alex Stuart
• Pål Axelsson
• Wolfgang Pempe
• Jon Agland
• Alan Buxey
• Mark Williams
• Anass CHABLI
• Christopher Whalen
• Rhys Smith
• Halil Adem

Goals

• Define next steps for discussion.

Discussion items

Draft minutes:

Review after last meeting

Nicole gave a brief overview of the last Baseline meeting, see: 2019-11-25 Baseline Expectation Meeting. 

eduGAIN SAML Profile compliance

When thinking about the net impact of making changes to the baseline used within federations / eduGAIN, we can learn from previous experience.  The eduGAIN validator v.2.is checking both MUSTs and SHOULDs. It's clear that when we implement MUSTS, it is possible to get federations to be meet compliance and gives them an incentive: https://technical.edugain.org/compliance_audit.  SHOULDs are not likely to effect change.  Most of the SHOULD failures within the validator at the moment relate to MDUI issues for individual entities.  Currently one federation signalled as RED (failing MUSTS) and 32 as AMBER (failing SHOULDs).

Previous conversations around entity inclusion in metadata have focused on "opt-in" vs "opt-out" requirements.  This is not the right conversation to be having anymore.  Each entity should be curated into eduGAIN based on a quality threshold.

What would be the net impact of removing these entities?

We need to think about how long will we give a Fed to mitigate issues.  A similar process to the last update to SAML profile would be needed.  Should an entire Federation be punished when only one single entity is failing?

We would also need to think about what do if an entity that previously complied suddenly started having problems. 

It's clear that only the offending entity should be removed, but the preference would be to work with the federation to remove the entity from eduGAIN rather than having eduGAIN OT remove.  This would need cooperation of federations. 

Attendees were keen on a "dashboard' type approach for entities and federations to be able to do a healthcheck that pulled in all the information that is currently in various different tools. This could be looked at by: https://wiki.geant.org/display/gn43wp5/Automated+notification+of+issues+and+overall+status+for+federations+en+entities.  Note for Niels van Dijk.

Pal raised a focus on setting baseline that was about increasing trust and ensuring service functionality rather than convenience.  An example of convenience would SP logos in MDUI.

Test entities were raised as an important entity type in eduGAIN that might not be able to reach the new threshold.  There are various approaches to supporting this:

• ignoring entities with hide from discovery from audits;
• creating a new entity type to support test;
• only permitting test entities in local federations.

This is not seen as a significant barrier. 

What is the best mechanism for taking this forward?

There are several different ways that this work could be taken forward.  Nicole asked Tom Barton to explain why InCommon had chosen to have a series of high-level statements for their baseline rather than explicit implementation directions (e.g. adopt Sirtfi).  Tom explained this was to enable a staged roll-out of implementation that saw incremental improvements. 

This divided approach could work well in our current environment; undertaking the work on the high-level statements that should not change for long periods at REFEDS and supporting both federations and eduGAIN with implementation plans around the statements that are focused on more specific goals.

1. High level statements on REFEDS level.
2. Work separately with eG on the implementation.

A formal working group will be set up at REFEDS to solidify the high-level statements and potentially some example implementation plans.  A discussion can then be held at the T&I Town Hall on how to implement this for eduGAIN. 

Is this definitely a baseline, or a second tier?

The group agrees that it needs to be a baseline. Otherwise nothing will happen.

AOB

Request to set up regular meetings.

Concern reflecting on the funding issues identified by REFEDS.  Do federations have the ability to implement another bar raise?  Nicole to look at comparison of funding, staffing, entities and compliance in federations.

Action items

• Nicole Harris to propose formal REFEDS Working Group on Baselining.
• Nicole Harris and Casper Dreef to start a collaborative document for the high-level statements for REFEDS.
• Nicole Harris to look at producing a graph on funding, staffing, entities and compliance in federations.
• Casper Dreef to schedule a series of meetings for the baseline group.