Person Search Index From Delius
Outbound service, based on Logstash, that sends details of people on probation held in the Delius database to OpenSearch. The resulting OpenSearch index supports searching and matching details of people on probation and their contact records. This makes up the core of the Probation Offender Search service.
HMPPS Business Need
Searching and matching people on probation is needed by multiple systems within HMPPS. This service provides full and incremental indexing to ensure that we provide search and match capability for people on probation using an up-to-date dataset. The index is refreshed on a daily schedule and incrementally updated during the working day based on Delius data events.
Searching for contacts is needed by the NDelius system to support the “Free Text Search” screen. This screen allows probation practitioners to find relevant case history records based on given keywords or semantic similarity. The index is refreshed on a weekly schedule and incrementally updated during the working day based on Delius data events.
HMPPS Domain Mapping
The person search index contains the elements of Core Person Record that are currently stored in the Delius database. This includes information on their relationship with the probation service (Person OMiC/POM/COM), the* Person Location* and overview data on the person’s Sentence/Offence.
The contact index contains data that spans across multiple HMPPS domains, due to the nature of how contacts are used in Delius. For example - contacts are used for supervision appointments, drug tests, unpaid work. See Concepts#Contact.
Context Map / Interfaces
People on probation
The service takes data from the Delius database and converts to the document format that forms the ‘offender’ response of Probation Offender Search. This document format is the de-facto representation of the person on probation as used by HMPPS Digital services. The data format can be seen in the response definitions of Probation Offender Search The mappings between the indexed document data format and the Delius database fields are within the Delius database query that forms the core of the indexing process. The search and match functionality of Probation Offender Search is determined by the index template and the indexing pipeline which together define the pre-processing and field analysis that result in a searchable index.
Contacts
The service takes data from the Delius database and converts to the document format that forms the ‘contact’ response of Probation Offender Search by the Delius database query and the Logstash full-load and incremental pipelines.
Data is ingested in two formats - as lexical keywords, and as vector embeddings (generated by a language model). This enables a hybrid lexical/keyword and semantic search over the contact list. See Hybrid Search with Amazon OpenSearch Service.
The ingestion and search functionality is defined by the following OpenSearch configuration files:
Processing Pipeline
Downstream Business Functionality
- Search of people on probation in HMPPS Digital services including NDelius
- Matching of people within HMPPS Digital services
- Matching of people to determine links between prison and probation systems
- Semantic search of contact list for a person on probation in NDelius
Relevant HMPPS Domain/Data Events
Incremental indexing is supported by Probation Data Events