Multiple effectiveTimes in Delta files - are they required?

Hi everyone

In the March 2020 US Edition we experienced for the first time a situation where multiple effectiveTimes were generated in the (Inferred Relationship) delta. This was a valid scenario because the Relationships were created as new in the Jan 2020 International Edition, and then correctly removed by the classifier in the March 2020 US Edition - thereby providing 2 records (with different effectiveTimes + Active flags only) in the US Edition Delta file. For example:

id	effectiveTime	active	moduleId	sourceId	destinationId	relationshipGroup	typeId	characteristicTypeId	modifierId
12058138020	20200131	1	900000000000207008	127259005	128133004	0	116680003	900000000000011006	900000000000451002
12058138020	20200301	0	731000124108	127259005	128133004	0	116680003	900000000000011006	900000000000451002

The obvious solution is to simply code systems to take the record with the latest effectiveTime for that ID within the Delta - however as this issue has now highlighted, some vendors have already hard coded theirs to take all Delta records regardless. For now we have advised them that this is a valid (though unlikely) state, that will only happen in our only Managed Service Edition (US).

However the question now is, once we move to continuous delivery, do we still need to include multiple effectiveTimes in the Delta files that people will be able to produce for themselves, using the new Delta generation tool?

Or, are we happy with always taking the latest state in the Delta, regardless of how many (Published) changes there were in the meantime?

So, for example, if the following record is changed 3 times between January 2022 and July 2022:

id	effectiveTime	active	moduleId	sourceId	destinationId	relationshipGroup	typeId	characteristicTypeId	modifierId
12058138020	20220131	1	900000000000207008	127259005	128133004	0	116680003	900000000000011006	900000000000451002
12058138020	20220431	1	900000000000207008	127259005	128133004	1	116680003	900000000000011006	900000000000451002
12058138020	20220731	0	900000000000207008	127259005	128133004	0	116680003	900000000000011006	900000000000451002

Should the Delta (for someone who is currently baselined on Jan 2022) generated for Jan 2022 - July 2022 include

a) ALL changes (including multiple effectiveTimes):

id	effectiveTime	active	moduleId	sourceId	destinationId	relationshipGroup	typeId	characteristicTypeId	modifierId
12058138020	20220431	1	900000000000207008	127259005	128133004	1	116680003	900000000000011006	900000000000451002
12058138020	20220731	0	900000000000207008	127259005	128133004	0	116680003	900000000000011006	900000000000451002

b) ....Or should it only contain the latest state:

id	effectiveTime	active	moduleId	sourceId	destinationId	relationshipGroup	typeId	characteristicTypeId	modifierId
12058138020	20220731	0	900000000000207008	127259005	128133004	0	116680003	900000000000011006	900000000000451002

When we originally all discussed this we were all in favour of a) (including all published changes in the interim period), as this follows the current RF2 spec, and allows people to generate the correct Full file even if they only ever take Delta's, and regardless of how long they wait inbetween each Delta.

However, this latest discussion has brought into question whether or not there are actually any valid use cases for this?

Do any users (certainly down at the vendor level) actually care about the interim history, or do they only ever want the latest state?

Those generating Delta's could just take the latest Full/Snapshot from the latest published release (and it will contain all changes), and those not using the Delta's will be unaffected.

One of the few use cases we can envisage is for historical ECL, but it sounds like the complexity involved here might require a rather different solution anyway, so that might not be a strong enough use case to require all vendors to change their systems to cope with multiple effectiveTimes?

So, if possible can everyone please post known use cases for including all changes (option a)) against this blog, and when we have our next TRAG call we can discuss in detail?

Thanks very much!

Andrew