The following section outlines the incremental phases undertaken to design and implement UniSQ’s Evaluation System:
Phase 1: Impact Evaluation of Academic Quality at UniSQ In 2019, an independent principal evaluator conducted an impact evaluation, engaging key stakeholders at all levels of the learning experience. Through 57 interviews with executives, heads of school, program coordinators, associate deans (learning and teaching), course coordinators, and students, valuable stakeholder input provided a deep understanding of the current state of academic quality at UniSQ. This evaluation served as a foundation for subsequent phases of the project.
S3-19 Pilot Evaluation Results
Phase 2: System Planning, Piloting, and Collaboration with Stakeholders Following the impact evaluation, preliminary plans for the Academic Quality Framework system were developed and shared with stakeholders. Collaborative workshops were conducted to gather extensive feedback and actively involve stakeholders in the development of process documents. UniSQ’s Evaluation System design, focusing on accountability, improvement, performance, and investment domains, was refined through a pilot phase involving five schools in 2019 and a second pilot in 2020. Feedback from staff and students informed adjustments to system components and operation protocols, resulting in a more effective and contextually relevant system.
To support the integration of current and new data systems, a workflow diagram, depicted in Figure 1, was developed. This diagram served as a visual representation of the course enhancement conversation process, outlining the various steps and interactions involved. This diagram was instrumental in guiding the implementation of the process during the university-wide pilot of course enhancement conversations in Semester 1, 2020, following a limited pilot conducted in Semester 3, 2019.
Phase 3: System Implementation and Action on Outcomes In 2020, UniSQ’s Evaluation System was implemented across all schools. Over the next two years, feedback from student responses and professional development sessions with stakeholders was collected. An analysis of the system’s impact on student success and academic support was conducted, combining quantitative insights and qualitative thematic analysis. This evaluation highlighted the need for further capacity building to support program and discipline development. Notably, three key innovations emerged during this phase: qualitative evaluative analysis support through environmental analytics scan reports and growth share matrices, quantitative integrative program evaluation reports including curriculum structures and models, and the UniSQ Graduate Employment Program.
Phase 4: System Checking, Consultation, Evaluation, and Revision Since 2023, the Evaluation System has undergone a rigorous process to develop university policy documents that ensure ethical use of the generated data. Efforts have also been made to expand policy inclusion for underrepresented stakeholders and implement structured practices to address identified challenges in a systematic manner. Ongoing evaluation, consultation, and revision play integral roles in this phase, enabling continuous improvement and refinement of the Evaluation System. Capacity development workshops have been conducted across all schools and support units, fostering a collaborative and community-based approach to evaluation. Ethical and effective use of evaluation processes and data has been enhanced through the development of protocols in collaboration with staff. Various professional development sessions and consultations have been held with academic staff members and student-related service units, promoting knowledge sharing and engagement with the Evaluation System.
Throughout these phases, UniSQ’s Evaluation System has evolved and matured, driven by stakeholder input, rigorous evaluation practices, and a commitment to continuous improvement. The collaborative and consultative approach has ensured that the system aligns with the needs and aspirations of the university community, resulting in an effective framework for enhancing academic quality and supporting evidence-informed decision-making.
