MATC Matters

Looking at AQIP
Category 7: Measuring Effectiveness

AQIP has identified nine categories of related key processes that provide a framework for the assessment and improvement of higher education institutions. The seventh category, AQIP Category 7-”Measuring Effectiveness,” examines the processes and systems related to the gathering, storage, management, analysis and use of data as well as comparative data to measure the effectiveness of continuous improvement efforts.

The Systems Appraisal Team identified key characteristics of the College that are relevant to Category 7, “Measuring Effectiveness” including:

• The College as a large and comprehensive technical college has many avenues and uses for data collection.
• MATC has implemented the PeopleSoft portal and, as the lead college of the Southwest Consortium, is continually working on improving data access and use that support user’s continuous improvement efforts.
• The College is committed to increasing access but in order for this to occur it must also respond to market shifts, changing stakeholder expectations, increasing competition, decreasing market penetration, increasing decision-making limitations due to WTCS policies, and organizational barriers. In this environment, data gathering, storage, management, analysis, comparison, and use will be vital for supporting growth and meeting current and future challenges.

Items which the College monitors and measures on a regular basis include:

• Feedback from stakeholders regarding data integrity, reliability, accuracy, and availability.
• Data usability, accessibility, and process satisfaction results from the PACE survey
• Data gathering, management, and satisfaction results from the Constellation survey
• Results of the annual Institutional Research Office user’s survey
• Results of the annual LSQIPS user’s survey

The Systems Appraisal Team identified many strengths related to “Measuring Effectiveness.” The following list includes some of the more important strengths identified:

• MATC selects, manages and uses data to support five primary processes-the Environmental Scan (Institutional Research and Effectiveness), the AQIP Action Projects, College Work Teams, LSQIP, the College’s program review process and Learner Success Programs.
• MATC collects and analyzes data from multiple measures including several different satisfaction surveys used with different stakeholder groups, Strategic Plan goals and objectives results, LSQIP achievement level measures, PACE results and the AQIP Constellation results.
• The College uses multiple means to identify new and existing needs for data by different stakeholders.
• MATC is beginning to identify and use comparative data and working to incorporate relevant data from local and national sources.
• The College also is beginning to implement a system with clearly defined connections and responsibilities that have the potential to align student learning and data analysis with stated key institutional initiatives.
• The development of a balanced scorecard for business operations and equity scorecard for diversity-related issues. The Systems Appraisal Team noted that this model could be expanded to other areas of college such as student services.

The Systems Appraisal Team also provided feedback on opportunities for improvement that included:

• The College use of multiple means to collect data but heavy reliance on anecdotal indicators of improvement. The Team recommended a future focus on instilling the importance of measuring effectiveness amongst targeted employee groups.
• The College bases its current results on internal and system-wide findings but would benefit from making outside comparisons to similar institutions through integrating comparative data from resources such as the National Community College Benchmark database.
• The College is continuing to work on improving assessment of its measuring systems. The Team recommended that the College focus on making information clear and simple for end users and more clearly identify performance targets to aid in prioritization of efforts.

The gathering, management and use of data are integral to all Action Projects. Although no current Action Project is specifically related to “Measuring Effectiveness,� the College has retired two projects in particular that focus on data collection and use.

• Market Research-programs and services are developed and improved based on data focused on customer needs. This Action Project resulted in the current Enrollment Management Plan.
• Measuring/Process Integration-creation of an Integrated Reporting System (IRS) to provide a web-based, interactive data warehouse that will allow all employees to make data-driven decisions. The College retired this project after a number of technological, personnel and process issues created prolonged delays. It is the intention of the College to revive this Action Project when sufficient resources are in place that allow for a greater chance of success.

Measuring Effectiveness is a responsibility every employee should take seriously. Knowing how effective our efforts have been helps us to close the continuous improvement loop and allows us to align our efforts with the goals of the College. While the gathering of relevant data is an important part of continuous improvement, it is only one part of the equation. Using data through analysis, making comparisons and applying the newly gained knowledge supports not only our individual goal attainment but supports the College in achieving its strategic goals.