We provide a comprehensive set of evaluation services, including designing data collection protocols, administering surveys, conducting interviews, and shadowing project activities. We lead in analyzing the evaluation data collected, including transcribing and coding interview data. We create data visuals and prepare evaluation reports and presentations for sharing findings with funding agencies and advisory boards.
1. Logic Models / Theories of Action
We can prepare a logic model/theory of action related to your team’s project design. This document is typically included in an initial proposal or an appendix of a competitive renewal.
2. Evaluation Frameworks
We can prepare an evaluation framework based on your team’s logic model/theory of change. The framework is often used to guide the overall evaluation and can include:
1) Evaluation questions to be answered.
2) Appropriate indicators / measures to gather as evidence in answering the evaluation questions.
3) Data collection methods and information sources to be used in the evaluation process.
4) An appropriate timeline for data collection.
3. Development of Evaluation Instruments
We can collaboratively work with your team to design instruments for data collection aligned with the evaluation framework. This collaborative process often includes:
1) Sharing a custom-built library of items with your team to review and select preferred items.
2) Reviewing your team’s selections and assembling draft protocols for use.
3) Sending Word-based drafts of instruments for review.
4) Alerting your team of gaps based on the evaluation plan and making recommendations for final the drafts.
5) Finalizing draft of protocols and programming surveys for web-based administration.
4. Surveys
We can collaborate with project leaders to establish web-based surveys and/or feedback forms. These surveys are often hosted via our SurveyMonkey or Qualtrics accounts.
6. Site Visits
Your plans can include budgets for our team to travel to your site. During our visit, we can collect survey data, conduct focus group sessions, meet with project leaders, and / or observe events. Our site visits often close with debriefing sessions that alert project leaders of preliminary findings and recommendations for improvement.
7. Event Shadowing / Observing
We can shadow key project events. The shadowing / observation process is typically anchored by detailed note taking, whereby each evaluation team recorded notes roughly every 3-5 minutes during the event and/or when there is a major transition in the agenda (e.g., switch of topic or speaker). Our reports from shadowing events often include:
8. Partner / Program Team Assessments
We can work with your team to assess the health of your partnership/program team using a validated Partnership Self-Assessment Tool. The tool measures your partners’ thoughts on your team’s: level of synergy, leadership, efficiency, administration and management, non-financial resources, and financial resources. Descriptive data tables of aggregate survey results (without identifiers) are typically shared with your team, along with recommendations on partnership strengths and weaknesses.
9. Teaching Appraisals
Your team can contract with the evaluation team to provide written and confidential feedback to instructors who teach courses as part of your project efforts. This process can be in addition to the already establish course evaluations in place at your institution, but can include items that are better aligned with outcomes promised within your proposal. The evaluation can work with leaders to administer the form at the close of every semester courses are taught. The team at SEI can analyze these data, prepare individual summary files for each instructor's private review/reflection and aggregate the set of evaluations to contribute to the broader program evaluation.
10. Concept Inventories
We can assist your team in organizing/data entry of pre- and post- concept inventories that are internally developed. Your team can internally administer, score, and record results for our team to analyze.
11. Analysis of Internally Collected Documents
Your evaluation budget can include resources for the our team to analyze data that are internally collected. These data can include:
1) Recruitment records.
2) Analytics collected by the project leaders or stakeholders.
3) And/or internally collected attendance and feedback on services/activities.
12. Formal Summaries and Reports
We can prepare reports detailing evaluation activities and findings that have constituted the contract period. Our reports often include recommendations from stakeholders for improvement.