Sentinel indicators are utilised to support program quality for undergraduate and postgraduate courses and programs.
The concept of the sentinel indicator is inspired by ecology, where certain species are considered “sentinel” species that provide early warnings of issues within an ecosystem. Similarly UniSQ views learning and teaching as a complex ecosystem, where student evaluation data is just one component. This approach aligns with the understanding in academic literature that student evaluations should be part of a comprehensive evaluation framework.
The use of a sentinel indicator, as opposed to a performance indicator, is essential to ensure that all data points are considered within the broader context of course data. Sentinel indicators do not provide a final evaluative determination but rather indicate the need for further investigation into the course or program context to identify potential risks to quality.
Sentinel indicators serve as early warning signs, triggering program and course enhancement conversations. They are contextualised within the broader course data and comprehensive evaluation framework to ensure fair and meaningful discussions focused on continuous improvement and enhancing the quality of education at UniSQ.
Sentinel Indicators for Course Quality
To reflect the risk-based approach to curriculum quality assurance, the Sentinel Indicator score for Course satisfaction in the Student survey (at both mid and end-Study Period) which triggers a Course Enhancement Conversation is:
- for the years 2020-2021, less than or equal to 3.5 with 5 or more responses
- for the years 2022-2023 less, than or equal to 3.6 with 5 or more responses
- for the years 2024 onwards, less than or equal to 3.7 with 5 or more responses.
Sentinel Indicators for Program Quality
To reflect the University’s risk-based approach to curriculum quality assurance, Sentinel Indicators which trigger a Program Review Conversation are as follows:
- Undergraduate EFTSL <30
- Postgraduate EFTSL <15
- Commencing and continuing attrition
- Undergraduate attrition ≥20%
- Postgraduate attrition ≥10%
- Progression rate <80% (Australian Government benchmark)
- Completions
- Undergraduate completion count <30
- Postgraduate completion count <15.
Focusing on student load, attrition rates, progression rates, and completions. Programs meeting any of these indicators are required to report to the Education Committee and Academic Board, entering into a Program and Study Components Review Conversation with the Evaluation Methodologies team.
Impact on quality outcomes over time
In both Semester 1 of 2022 and 2021, 36 courses met the sentinel indicators. The mean score for the question “I am satisfied with this course so far” was 4.06 in Semester 1, 2022, for courses with five or more responses. A new emerging theme from the course enhancement conversations in 2022 was the higher levels of dissatisfaction expressed by students regarding the online learning experience. End-semester survey means have remained stable, with overall satisfaction remaining above 4.05 for surveys with five or more responses in both Semester 1 and 2.
At the end of every Course Enhancement Conversation participants are invited to conclude if the sentinel indicator was ‘True’ with multiple lines of evidence indicating issues in the environment or ‘False’ there were no further indicators in the environment or the data to suggest any ongoing issues. In May 2023, a review of the sentinel indicator where a mean of the student satisfaction would trigger a conversation. A review of 3.5 years of data indicates confirmation that 94% were ‘True’ with the remainder of 6% confirmed as ‘False’. This outcome would indicate that the sentinel indicator is a reasonable proxy to initiate a conversation to unpack the contextual variables.
