The Engagement Data Imperative – From Access to Action

Lynn Brabender (AASCU)

Coming from a data infrastructure (specifically a student success perspective): institutions are data driven, but it requires building data governance infrastructure, data collection infrastructure, etc. and begs the question: Do we have enough data?  The data is usually there, but the structures and processes aren’t usually there. For example, the data might not be incomplete, but can’t be disaggregated, unable to answer certain questions. AASCU works with institutions to disaggregate data via race/ethnicity and to use it to make change on campuses. Strategically collecting information and giving to administrators pushes for a continuous improvement cycle, recognizing that structures/processes need to be put in place to track progress toward equity goals.

Valerie Holton (CUMU)

We see it on the cover of the Chronicle/NYT, while higher education works to reclaim its mission for the public good, and we see that this goal is difficult to achieve and assess. While we see Community Engagement incorporated in strategic plans, mission statements, Carnegie, and other rankings/classifications, consistently there is an inconsistency of methods and measures of what is being tracked. Collaboratory helps us have a somewhat standardized conversation around this data. We ask: Should data be centralized or decentralized? For some institutions, a centralized data collection system doesn’t make sense (too much infrastructure required, too complex, too many resources needed). Most data tracking efforts live in Community Engagement offices, but most CE folks don’t have extensive Institutional Research training. As a field we are not entirely sure how to operationalize this work, how it relates to the local context. While we do have data, we aren’t always sure what to do with it and to be able to feed it into decision-making is a whole other thing that higher education is struggling with. 

Marisol Morales (ACE)

What really concerns me is about where does equity fit in terms of the data, both in infrastructure and the time to analyze the data. Where does this fit in with other institutional priorities and time/resources? Data technology has improved, but that also costs money. For institutions that are under-resourced, what kind of support are there to help all of our campuses? What is available to assist and help them pull the same kind of data so they can assess across regions, various campuses, etc. to have a more accurate picture of the Higher Education Community Engagement landscape. Institutions and organizations need the time to sit down with the data. Specifically the need to disaggregate data based on various types of student groups, particularly pulling the same kind of data to assess across campuses, regions, different types of institutions to have a more accurate pic of the landscape. We need the time to sit down with data. Reflection as a tool for people doing service and engagement, but we don’t always have that for ourselves. How can we take advantage of things like Carnegie to ask institutions to make the commitment to collect and assess the data? Particularly looking at the disaggregated data around race and ethnicity, we need to know the impact of this on different communities, particularly as we are sending a diverse set of students into communities that are much like their own. 

Lynn Brabender (AASCU)

Building cross campus student success teams that look at the data together, building systems that scaffold out into the campus and community, relying on mechanisms of goal setting and metric setting, communicating across campus. In advising, we’ve really seen institutions move the needle on data driven work, creating processes where they are continuing to track focused issues and revising policies to ensure they are making the necessary impact. 

Marisol Morales (ACE)

That makes me think about the California State System and a dashboard they have around student data, they can bring data to course-level and have conversations with faculty about what they are seeing and the ability to do this across the system is really powerful to think about how you make gains particularly for underrepresented students. 

Marshall Curry (EPIC-N) 

I am overall seeing universities in general are starting to want the collection and management in community engagement systematically, much more than they were ten years ago. What I’ve seen from our member programs, there are more calls asking your program is doing what in community…we need to show that. Please collect that data. We need more community engagement, let’s do an epic model program and track that. 

Valerie Holton (CUMU) 

I have a story to share: I used to work in homeless services, prior to what we now have is the Housing First model; and, I remember when the United Way asked us to provide a logic model. We had to tell outcomes and characterize why it was essential work. We thought we were providing needed programs to keep people off the streets, research on homelessness told them that length of stay in shelters had nothing to do with housing stability (it had much more to do with income and affordable housing). This example has shaped how I think about assessment that we are all trying to do. We are doing this because we think it is impactful. But we’ve seen across fields that we can be more impactful, that our work can be improved and impact more people in more ways. Taking time for assessment and evaluation shifts priorities away from doing the work but we need to reframe the necessity to do this evaluation as core to the work. 

Marisol Morales (ACE)

We need to be realistic that on some campuses positions have been eliminated or not filled and there are fewer people able to do the work, let alone take the time to collect data. We cannot continue to put more on people without putting the resources and investment in that. Campuses have utilized a lot of different tools – but not enough time in investing in people using tools. We need both. The conversation about capacity is one we have to have at the institutional level, particularly because we are losing so many people. For example, many of the people who worked on previous Carnegie classification applications are no longer at the institution. There has to be a way to talk about this, that brings in the human capacity, training and support to be able to do this work effectively, ethically. 

Marshall Curry (EPIC-N)

I see that so often, too, where the person being asked to do this is starting from scratch. Universities want to get more of the management and analysis of engagement, front and center, and we should use that to advocate for human capital to do that work. For example, asking the institution where they are okay to let the work fall off if assessment is the priority with the same human capacity. Bridging connections among colleagues/campuses helps with resource sharing, and sharing unique ways to do this work.

Lynn Brabender (AASCU)

I want to emphasize again the importance of having cross campus teams and the need to have ownership not just by Institutional Research; where we’ve seen this ownership extend throughout units, we’ve seen the democratization of data use/data driven work, helping to instill a data drive culture at the leadership level. This team should include institutional research, information technology, student affairs, academic affairs, faculty. One avenue is to roleplay in these cross campus meetings how people use data to answer certain questions. 

Marisol Morales (ACE)

Part of this work should include bringing in community members and training them on data collection and analysis, for example, there is a trend in health-based research that looks at participatory models – training community members to collect data. Another consideration is how to bring students along into the process. How do we also build interest by taking the show on the road, sharing the data that we’ve collected on and off campus? We need to use both qualitative and quantitative methods as a means to understand our work more deeply. 

Valerie Holton (CUMU)

By focusing on attracting students – students looking for meaningful ways to learn and be engaged in community – demonstrate this as a recruitment/retention tool, and for attracting faculty, particularly faculty of color and women, faculty with service learning experience, coming into academia wanting to leave an impact.

Marshall Curry (EPIC-N)

What I’ve seen following our association’s call to adopt SDGs, is members using this framework to characterize their community engagement. This has brought attention to their university, international attention. Arizona State University, recognized internationally in Times Higher Education Impact Rankings. They know how to tie SDGs – research, community partnerships – to the SDGS and get that recognition. The work is amazing, but some of it is how they are aligning their data with broader priorities. How do they use data with community partners? The community doesn’t need data on students. The community needs to know about community impact. If so, what is that impact? How do we help community members use frameworks and analyze their data, and make more data-driven decisions? They need help doing that, so can universities provide that help. A university in San Diego is helping to create a community map of the communities’ work regarding SDGs. Another example is the work with Kristin Norris and Indiana University where their Center for Rural Engagement, an EPIC-N program, measures the strength of relationships in the community using Collaboratory. It’s incredible work. 

Marisol Morales (ACE)

When I was at Campus Compact and doing the civic action plans, Loras College did a civic action plan in partnership with the city of Dubuque, so you see where this collaboration is happening. The important part about data is realizing when you need to pivot, when what you are doing is not working. Then you try different models, bring in different voices. 

Valerie Holton (CUMU) 

Encourage programming that supports this kind of work and share best practices via various mediums – conferences, journals, webinars, maybe the CUMU Journal. Also it is important to communicate with funders and ensure that there are realistic expectations for this work  – long term expectations, with funding allocated with that in mind.

Marisol Morales (ACE)

Being realistic, the same kinds of campuses continue to get the money from funders; whereas we should focus efforts toward various campuses that are not provided with the same resources to build infrastructure, like HSIs/HBCUs/Rural campuses. It is also important to acknowledge that colleges harmed communities in the past, needing to ethically move forward. Data collect and storytell with the community, acknowledging that there are multiple sources of knowledge and data comes from various mediums. Tell a more holistic story, putting us on a path for truth telling, stories of impact and change, hurt and reconciliation, stories of transformation.

Lynn Brabender (AASCU)

I would build off of what Valerie mentioned. At AASCU, we are working with various presidents and provosts and are working on various programs to ensure that they are data literate and promoting data transparency; and but, similarly funders can drive the value of data, particularly toward continuous improvement; and can be there to support resources that area already there, emphasizing on reporting metrics, supporting under-resourced institutions along the way.

Marshall Curry (EPIC-N)

Really focus on the members doing the work on the grounds, taking best practices and making them easier to use. We have taken assessment and other resources from journals, repurposing to decrease barriers. We can do those steps to decrease the entry barriers. By working together, what is the global data literacy certification that we all adopt? Can we get close to one instrument for this? Maybe not, but do we need the systems/funders/associations working differently? Using our space and funding opportunities to assist with evaluation and collect stories of success, i.e. this happened, helping the community and students figure out what to do or not to do. 

Marisol Morales (ACE)

It takes time to realize the impact; sometimes seeing the full impact takes a long time, could be a 10-15 year project. For example, the “aha” moment that individuals/organizations could have years after. These impact stories need to be understood within that context and what can be measured in between.