Client Outcome Measures (COMS)

A project advisory committee was formed in late 2008, consisting of representatives from the NADA membership, NSW Ministry of Health (MHDAO and InforMH) and external experts in the areas of research, data management, mental health and drug and alcohol policy and service delivery.

To inform the selection of items and measures in the data set, NADA undertook two major activities.

1. A researcher was engaged to undertake a critical review of screening, assessment and outcome measures that may be used in drug and alcohol service delivery. This review resulted in a report entitled A Review of Screening, Assessment and Outcome Measures for Drug and Alcohol Settings. This report is available on the NADA website.


2. NADA conducted a baseline consultation questionnaire with members to gather information on the standard measures currently used in the non-government drug and alcohol sector, and determine how client data is collected and used by those services.

The results of the baseline consultation indicated that:

• A variety of standardised measures were in use across the sector


• Most organisations were not regularly completing an outcomes measure, i.e. a standard measure that is taken at various points during treatment to measure progress and outcomes and to inform treatment planning.


• Most organisations would like to systematically collect more information on the impacts of drug and alcohol treatment on clients.


• There was broad agreement that enhancements to the current NADA on-line database and the implementation of standardised client outcome measures was the best way to improve client information management in the non-government sector.

• What is the tool designed to measure?


• Is the measure appropriate for use in a range of service types, with a range of client groups and by a range of staff with varying levels of experience and expertise?


• Does the tool require an initial cost or ongoing licence fee?

The COMS as a whole needed to be comprehensive enough to cover the range of client outcomes that can result from treatment and brief enough to be used without imposing a large, additional data collection burden. The COMS could replace some existing measures used by NADA members and/or reduce the need to collect a number of other measures. COMS had to be applicable across a range of services and program types (e.g. residential facilities, counselling and case management programs).

The project advisory committee made final recommendations on the items for the outcomes data set, incorporating existing standardised measures and items from tools that have undergone research to establish their validity and reliability.

The following four outcome domains form the COMS Outcome Measures.

1. Drug and Alcohol Use: Severity of Dependence Scale (SDS), Drug and Alcohol Use – BTOM and AATOM items on frequency and patterns of use.

2. Psychological Health: Kessler 10+

3. Health and Social Functioning: WHO EUROHIS QoL-8, 3 NSW MDS ietms on living arrangements and income status, 2 BTOM items on crime

4. Blood Borne Virus Risk: 4 items on injecting drug use and overdose from the BTOM-C

Drug and Alcohol Use

Severity of Dependence Scale (SDS)

• The SDS is a brief five-item screening measure of psychological aspects of dependence.  This short (it contains only five items and takes less than one minute to complete) yet effective tool is easy to administer to evaluate the level of severity of substance dependence perceived by the client.


• The items are specifically concerned with an individual’s feelings of impaired control over their drug taking and with their preoccupations and anxieties about drug taking.


• It is widely validated across a range of drug using groups, including heroin, cannabis, cocaine, amphetamine and benzodiazepine users. Studies among heroin, amphetamine and cocaine users have shown the SDS to be a reliable measure of psychological dependence.


• The SDS has been validated in a range of translations and in both adolescent and elderly populations. It has also been used with Aboriginal and Torres Strait Islander peoples in research studies.  The wording of the SDS is straightforward and the concepts appear to be understood by a variety of different groups of drug users.


• SDS scores range from 0-15 with higher scores indicating a higher level of dependence.


• Anecdotally, researchers who have used the SDS have found that scores may stay elevated for a period of time even after an individual has stopped or decreased their drug use. This could be related to the fact that the SDS questions focus on concern/worry about drug use, rather than the actual amount used or whether the person used at all.

Drug and alcohol use (Items from BTOM and AATOM)

• Drug and alcohol use is measured by asking:


• Number of days used for illicit drugs and benzodiazepines, and


• Number of days used and quantity used for alcohol and tobacco.


• Two separate measurements for alcohol use were included: number of days the person drank alcohol and average number of drinks per day, and number of days of heavier drinking than usual and average number of drinks on those days.
  
  

  Psychological Health

  
  

  Kessler 10+

  
  


• The Kessler 10+ scale (K10+) is a widely used, simple self-report measure of psychological distress, which can be used to identify those in need of further assessment for anxiety and depression. It is based on questions about the individual’s level of nervousness, agitation, psychological fatigue and depression in the past 4 weeks.


• It is designed to span the range from few or minimal symptoms through to extreme levels of distress. This measure was designed for use in the general population to detect high-prevalence mental health disorders (e.g. anxiety and depression); however also serves as a useful clinical tool and scores may be an indicator of mental health disorders with lower population prevalence (e.g. schizophrenia and Bipolar Disorder). The K10+ can also be used as an outcome measure. Changes in K10 score can be strongly indicative of both improving and worsening general psychological distress as well as a warning sign of deterioration of a clinical mental health condition.


• The K10+ has been shown to be a very good screening tool for detecting levels of distress that are associated with an independently determined current DSM or ICD diagnosis of an anxiety disorder and/or depressive disorder, and has been found to outperform other instruments in detecting anxiety and depressive disorders.


• The K10+ has been used in a wide variety of surveys including the Australian National Survey of Mental Health and Wellbeing, surveys in New South Wales for Chief Health Officer and surveys of Aboriginal populations in Western Australia and the Northern Territory.


• The K10 is commonly used and understood by non-mental health specialists such as General Practitioners.


• The K10 has been successfully used in a range of populations, including a range of different Australian settings and specifically with drug and alcohol users in Australian settings. It has been used in and translated into a number of languages other than English and validated with a number of cultural groups.


• This measurement tool consists of ten core questions (1-10). Each item is scored from 1 to 5, from “none of the time” to “all of the time”. Scores are then totalled, resulting in a K10 score between 10 and 50, with higher scores on the K10 indicating greater distress. Missing items are excluded from the calculation of the total score.


• The K10+ includes an additional four questions (11-14) that aim to quantify the impact or degree of disability associated with the person’s identified degree of psychological distress. Note that items 11 to 14 are excluded from calculation of the total score.


• Although these additional items do not contribute to the total score, they assess variables that give an indication of the impact or degree of disability associated with the person’s level of psychological distress.
  
  
  
  • Questions 11 asks consumers to identify how many days in the last month they were TOTALLY UNABLE to function, while Question 12 asks of the remaining time in the last month, how many days did they have to CUT DOWN on activities of daily living as a result of their distress.
  
  
  • Question 13 asks respondents to identify how many times they have had to consult a health professional in the last month. Note that the maximum number of consultations allowed is 89 or almost 3 a day!
  
  
  • Question 14 has respondents indicate the amount of time their psychological distress was related to physical health problems rather than mental or emotional distress.
  
  
  
  


• Scoring the K10 is simple. The administration, scoring and interpretation of the K10+ needs no special training and n

Health and Social Functioning

WHO-8: EUROHIS Quality of life scale

The World Health Organisation Quality of Life 8 questions (WHO QoL-8, also known as the EUROHIS QoL-8) was designed for use as a very short and concise quality of life instrument. The WHO QOL 8-item index was developed as an adaptation of the WHOQOL-100 and the WHOQOLBREF and is therefore an international cross-culturally comparable quality of life assessment tool. It has been used and validated across a range of populations, including in drug and alcohol settings and with those with mental health disorders.  It has been validated cross culturally and in the Australian context. It has been found to be reliable and valid in both older and younger people and has been extensively tested in both psychiatric and drug and alcohol contexts.

• The WHO QoL-8 is a broad domain based measure that has applicability across the range of program types in the NGO drug and alcohol sector in NSW.  It measures quality of life across the following domains:


• General or Overall Quality of Life


• Overall perception of health


• Quality of physical life


• Quality of psychological life


• Quality of social relationships


• Quality of living environment

NSW MDS items

• 3 items from the NSW MDS have been included. They relate to living arrangements and income sources, and are a different and more objective measure of changes in these areas than the satisfaction ratings from the WHOQOL-8.


• Three items ask about a person’s living arrangements, who they live with and their main source of income.  These items would be collected by most NADA members as part of their NMDS or MDS data collection taken on admission and reported on through closed treatment episodes; however they also form a useful outcome measure and can be taken at a number of occasions during a person’s engagement with a service.

BTOM-C items on arrests

• The number of arrests is an important outcome measure, as crime may be instrumentally linked to the funding of drug use, and a reduction in criminal behaviour is an important societal and personal benefit of treatment. The 2 questions on arrests were regarded as the most appropriate outcome measure of crime, due to the difficulty (including ethical and confidentiality difficulties) of collecting information on specific criminal activities through self-reporting.


• The questions are the crime questions that are part of the validated BTOM outcome measurement tool developed by NSW Health.  These two questions ask the number of times the client has been arrested in the last 3 months and how many arrests were for offences committed in the last 3 months.

BBV Exposure Risk Taking Scale

BTOM-C items on risky drug using practices

• Injecting drugs users (IDU) are susceptible to a number of harms associated with illicit drug use. The BBV exposure risk-taking domain comprises 4 items from the BTOM-C on injecting drug use and overdose. The questions are the risk taking questions that are part of the validated BTOM measurement tool developed by NSW Health. These items measure changes and outcomes in relation to injecting and other risky drug use practices.


• A key finding of the 2011 Illicit Drug Reporting System (IDRS) surveys reported twenty-five percent of participants had shared injecting equipment (excluding needles). Among this national sample, one in ten participants who inject drugs (PWID) borrowed needles in the month preceding the interview.


• Sharing of equipment such as needles and syringes, spoons and filters increases the risk of transmitting blood borne viruses such as Hepatitis C, HIV, and other injection-related infections. Drug injection trends among participants in the Australian Needle and Syringe Program showed hepatitis C antibody prevalence to be consistently high among respondents that reported last injecting heroin, cocaine, and methadone or pharmaceutical opioids. Other harms include fatal and non-fatal drug overdose, which is linked to a range of negative outcomes such as brain damage and even death.

The COMS can be directly administered by a support worker or clinician, OR can be completed by the client (with the support of a worker) on paper or directly into the COMS database.

• Ensure wherever possible the whole questionnaire is completed on one occasion. If this is not possible, record the date each section of the questionnaire is completed.


• If a person is self-completing all or part of the questionnaire, review the questionnaire to ensure that all questions are answered. If questions are unanswered, encourage the person to answer them.


• Ensure that all answers are based on the client’s response, not on the clinician’s guesses or assumptions.


• Assure clients that refusal to complete the questionnaire will not lead to their being treated differently.


• Ensure that the exact wording and format of the questionnaire (especially scored measures- the Kessler 10, SDS and WHOQoL–8) is adhered to, as the COMS is based on standardised measures.


• Assure the person that the questionnaire is subject to the same rules of confidentiality and privacy as all other information held in their client record.


• Inform the person that the information gathered will be used only for their individual treatment planning and will provide a useful tool to inform their progress; however that de-identified data may be used for organisation level planning and reporting, sector wide reporting or for research purposes.


• Explain to the person that identifiable information will be available only to those involved in their direct care.

Use Clinical Judgement

• If the person is distressed and completing the questionnaire may add to their distress.


• If the person is too unwell to complete the questionnaire.


• If the person is unable to understand the content and requirements for completing the questionnaire (e.g. due to a psychotic or mood disturbance).


• If there are cultural or language impediments that make self-reporting inappropriate.

The “DO’s” when administering the COMS questionnaires

• Do be warm, friendly and helpful.


• Do request and encourage the person to fill out the questionnaire.


• Do let the person know that you will be there to support and assist them if required.


• Do emphasize that there is not right or wrong answer.


• Do tell the person to answer the question based on what THEY understand the question means. Provide definition of words if the person is unfamiliar with it.


• Do encourage the person to answer ALL the questions.

The “DON’Ts” when administering the COMS questionnaires

• Do not force or command people to fill out the questionnaire.


• Do not answer the questions for the person.


• Do not tell the person how you feel they should answer the questions.


• Do not allow other people (including family members) to help the person complete the questionnaire (except in cases where issues such as literacy or difficulty with English are a factor).


• Do not minimise the importance of completing the questionnaire.


• Do not accept an incomplete questionnaire without first encouraging the person to fill out unanswered questions.


• Do not paraphrase, rephrase, interpret or explain a question.

While some questions in the COMS relate to a period of 3 months, the majority relate to a period of 4 weeks. This difference reflects the use of existing standardised measures that use these timeframes.

NADA has not specified or even offered a suggested timeline for the administration due to the diversity of organisations who will be using it. The time points at which to administer the COMS questionnaire will depend on the length of an organisation’s treatment program; the particular organisation’s needs and resourcing. However, organisations and services are strongly encouraged to apply the COMS as frequently as possible during treatment as this will allow for the most useful information both clinically and organisationally.

Timeframe Example:

A 28-day residential program might administer COMS at Intake and Exit and then follow-up assessment as part of an aftercare program. Alternatively, a 12-month residential program might administer COMS at Intake, progress stages every 3 months and at Exit. The follow-up questionnaire(s) might be completed as part of an aftercare program.

NADA members and users of NADAbase will generally be required to comply with the National Privacy Act including the 10 National Privacy Principles (1988- revised 2011) National Privacy Act and the NSW Health Records and Information Privacy (HRIP) Act (2002) NSW Health Privacy Guidelines. These Acts outline the responsibilities the organisation has with regards to collection, use and security of data.

Before administering any questions via NADAbase, it is important to inform clients that steps will be taken to protect the privacy of their personal information. The following are appropriate steps to ensure this. A sample Privacy and Consent Form is provided in the Appendix (Section 8).

Inform the client why you are collecting this information

Assure the person that their personal health information will be protected in accordance with the relevant Acts.

Inform the person they are entitled to access their own records according to your organisation’s policies and are entitled to make a complaint about handling of personal information and privacy.

Explain to the client that their personal information will be given to another person only if this is important for their health care or can be otherwise legally and ethically justified.

Explain that de-identified information may be used in service and sector wide planning, reporting or research activities.

Note: De-identified information is information or opinion about a person whose identity cannot be ascertained from the information or opinion. For information to be classified as de-identified it must not contain identifiers which, if linked with other information, could lead to the identity of a person.

Reasonable steps to de-identify the information – When de-identifying information, removing the name and address may not always be enough, particularly if there are unusual features, a small population, or there is a discussion of a rare clinical condition. Reasonable steps to de-identify might also include removing other features, such as date of birth and ethnic background that could otherwise allow an individual to be identified in certain circumstances.