Search and retrieval with accents (diacritic)
Currently, when you conduct a search the International Spanish Edition in the SNOMED International Browser using the diacritic (accent), it retrieves only terms without accents. When you conduct the search with an accent, it retrieved BOTH terms with and without accent.
In Snowstorm we have configuration to change how accented characters are handled. This is configured per language because each language seems to have different requirements. We have found that in some languages some accented/diacritic characters are considered as completely different characters and are included separately in the alphabet of that language. Whereas other accented characters do not appear separately in the alphabet and are just considered as modified versions of the original character.
This separation is important when it comes to search. For example in Sweden they consider characters 'å', 'ä' and 'ö' as unique characters which should never be simplified to 'a' or 'o' for search. This simplification is called character folding. In the latest Snowstorm release by default we are not folding/simplifying the characters: áéíóúüñ.
This means that these exact characters have to be used in the search term in order to match descriptions with those characters.
Questions for the SNOMED CT Spanish User Support Group:
1 - Is it okay to have the search conducted with or without accents and able to retrieve terms with or without accent across-Spanish dialects? Or would this need to be specific for each Spanish Extension shown on the SNOMED Browser?
2 - What characters should be and/or should NOT be folded/simplified?
3 - If you believe this needs a meeting/discussion please let me know.
If you could please leave comments below by Friday, April 23, 2021 so we can move ahead on this fix (if agreed by all). Thank you in advance for your help!
Corresponds to ticket: https://projects.jira.snomed.org/browse/BROWSE-415
Comments
Hi all,
We think it would be OK to have just one configuration for all spanish extensions, as far as we know there 's no more than one accent character for Spanish (´). It is totally necessary to have the search conducted without accent too), as even though it's a linguistic fault not to use an accent, in terms of usage it's a really common practice to browse terms without accent. We think correct spelling should be maintained for the retrieved answers though.
In Spanish we do not have folded characters, our only accent is considered as a completely different character. There's only one exception of a symbol called diéresis (ü), that goes over "u "letter, that may be considered as folded but it's used in really few words (ex. vergüenza)
We don't think a meeting is required for now, but thanks.
Uruguay NRCteam
Hi all,
In Argentina, we use snowstorm in the implementation and we have configured the service to be able to search with and without accent in the query, this was a demand from our users to have the same feature when using the browser.
1. We agree with the partners and believe that the search should ignore the accents and return the result anyway.
2. We don't have much to contribute to this point other than what Betania points out.
3. We do not consider a meeting necessary for this.
Argentina NRCteam
Though I'm quite strict with orthography, I think a search engine must be easy to use and should give all the intended results... and I say intended because not all the people know how to properly use accents in Spanish.
1. I think the search should ignore accents both in the search box and results.
As an example: if you search fémur or femur (incorrect)... it will give "fémur". And if you search fractura or fráctura (this is incorrect)... it will result "fractura".
You could also make a trigger/option available so the user could switch "accent exactitude" on or off.
2. I don't have an answer for this, as I think the search engine should be flexible on accents.
3. I don't think a meeting is required for this.
Regards,