PUB - Publikationen an der Universität Bielefeld

**Purpose:** Decision Support Systems (DSS) are not yet used in social work in Germany although large numbers of digital reports by social care workers are more and more available in institutions. Text mining methods have the potential to improve the decision-making of professionals in social work and to improve client lives by making more efficient use of available resources (cf. Coulton et. al. 2015, p. 10), i.e., in this case the huge amount of digital client records. There are no studies regarding the automatic prediction of the extra-linguistic attributes *level of need for assistance* (LONA) and *expression of volition* (EOV). Both attributes are linked to autonomy, which is highly valued in disabled people’s services in Germany.

**Methodology:** A framework is presented that can be applied for three use cases in disabled people’s services. Within the framework, text mining methods are used for the automatic prediction of the extra-linguistic attributes LONA and EOV in disabled people’s services using written reports. We describe a process of collecting a corpus based on data from two social care institutions in disabled people’s services: An annotation task is conducted to build datasets for classification tasks for LONA and EOV. To predict LONA and EOV automatically, classifiers are built with different machine-learning algorithms, including some state-of-the-art deep-learning algorithms (i.e., BERT, BI-LSTM, GRU, and CNN). The classifiers for LONA and EOV are evaluated in two settings, which are specific and agnostic to the area of life, which was assigned by social care workers for each written report. In addition, a method is presented to retrieve a dataset for the prediction of targets of EOV. A pattern-based approach is presented, yielding high precision in predicting EOV targets with lexico-syntactic patterns.

We developed a prototype DSS that relies on predictions of LONA scores from an SVM (one-vs-one) classifier. The prototype DSS is used as a stimulus in interviews to analyze the reactions of professionals to client trajectories based on automatic predictions, w.r.t. LONA, in the system to professionals responsible for social service planning (SSP) in disabled people’s services. The transcribed interviews were analyzed using qualitative content analysis, and the results are linked to the three use cases from the framework, to opportunities for further development, and to requirements to successfully implement such a DSS in an organization.

**Findings:** We built datasets for supervised multi-class classification of LONA (12,432 samples), supervised binary classification of EOV (10,268 samples), and evaluated models that are built with different machine-learning algorithms. The best results of the classification of LONA and EOV are obtained in a setting agnostic to the area of life. The best result for LONA is obtained with a linear SVM (one-vs-rest), yielding a macro F-measure of 94%. For EOV, a CNN yielded the highest macro F-measure of 98%. In addition, a training dataset with 2,283 samples and a holdout dataset with 1,549 samples in that targets of EOV are annotated are created. The training dataset is used to develop lexico-syntactic patterns that are used to predict targets of EOV in the holdout dataset. The novel pattern-based approach yielded a precision of 77.41%, an average overlap of 63.96%, and 25.93% exact matches on the entire test set. The applied BERT model outperformed our approach with an average overlap of 72.25%. In the analysis and evaluation of the single patterns that were applied in the test set, the results of our approach and the results from BERT were close. However, BERT still outperformed the pattern-based approach in average overlap with 57.65%. In comparison, an average overlap of 51.34% was reached in predicting EOV targets with lexico-syntactic patterns. This leads to the conclusion: BERT is better in predicting EOV targets, at least partially in unknown data, but with the pattern-based approach the predicted targets for EOV are more accurate.

The respondents in the user study saw opportunities to use the prototype DSS on the executive level (social care providers), operational level (institutional management and SSP), and on the level of social care workers. The respondents were interested in monitoring the independence of a client daily, as a complement to the yearly assessment of independence in SSP. Further opportunities were named that the DSS can be used for preparation to have more focused discussions with stakeholders. Furthermore, the DSS can be a help to write support plans. In addition, the DSS is not making any decisions itself and is a support tool that can be used at the professional's will. The expertise and discussions of professionals with the client and stakeholders are still very important.

**Value:** We investigated new tasks: the classification of the Level of Need for Assistance (LONA) and Expression of Volition (EOV), and the prediction of EOV targets in disabled people’s services. Because of the sensitive area that disabled people’s services represent, a supervised classification approach was followed to have a certain transparency in how accurate the predictions of the models are. The same applies to the pattern-based approach, which also yielded a high precision for the prediction of EOV targets. In domains such as disabled people’s services, precision is valued higher than recall to avoid incorrect classification that could have a negative impact on clients. There is also a potential to adapt the resulting classifiers and the pattern-based approach in other domains of social work. The framework presents possibilities (also regarding further attributes and data) on how to aggregate and visualize client trajectories in disabled people’s services. Furthermore, the prototype DSS is an exploratory example of how client trajectories could be presented to professionals in disabled people’s services in such a system.

*References*
C. J. Coulton, R. Goerge, E. Putnam-Hornstein, and B. de Haan, “Harnessing big data for social good: A grand challenge for social work,” American Academy of social work and Social Welfare, pp. 1–20, 2015.