National Center on Subsidy Innovation and Accountability Feedback Surveys

National Center on Subsidy Innovation and Accountability

Generic Data Collection Bank of Questions

Overview: This document, using constructs and format developed by the Office of Head Start and NORC to facilitate data aggregation and analysis across events of the National T/TA Centers and reviewed by the Office of Child Care (OCC), outlines a bank of data collection questions to assess customer feedback to technical assistance. It identifies five constructs and related survey questions to be used by the National Center on Subsidy Innovation and Accountability (NCSIA) to evaluate the delivery of universal, targeted, tailored, and intensive technical assistance (TA). These constructs are Content Quality, Content Fit, Increased Knowledge, and Intended Application of Knowledge.

This bank of questions is for poll, discussion, pre and post surveys and other customer feedback surveys for the NCSIA’s training and TA offerings. Most universal and short-term TA engagements will include a post-survey only. A pre-post survey will likely be used for intensive, targeted, and long-term individualized engagements. NCSIA will select the most appropriate questions from this bank of questions based on the specific technical assistance event.

All information collection requests will include the follow statement:

PAPERWORK REDUCTION ACT OF 1995 (Public Law 104-13) STATEMENT OF PUBLIC BURDEN: The purpose of this information collection is to obtain feedback regarding technical assistance received. Public reporting burden for this collection of information is estimated to average 3 minutes per respondent, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0401 and the expiration date is 5/31/2027. If you have any comments on this collection of information, please contact ncsia@wrma.com

Construct	Survey Question	Optional or Recommended	Responses	Example Type of TA for Use of Question	Alignment with NCSIA Logic Model
Content Quality	The content of the session was relevant to my work.	Recommended	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, effective and clear TA Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies
	The technical assistance provided was useful to my work.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, effective and clear TA Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies Lead Agencies report progress toward reaching their goals with the assistance of NCSIA
	The resources provided during the technical assistance/session/event were useful for my work.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, effective and clear TA Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies Lead Agencies report progress toward reaching their goals with the assistance of NCSIA
	The identified goals and outcomes of the technical assistance/session/event were met.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Targeted, Intensive	Desired Result: #3 – Effective and clear TA
Content Fit	Please let us know whether you found the content presented in this technical assistance/session/event to be too simple, too advanced, or just about right.	Recommended	Scaled item Far too advanced (1) A bit too advanced (2) About right (3) A bit too simple (4) Far too simple (5)	Individualized, Targeted, Intensive	Desired Result: #1 – Trusted partner
	What outcome or desired result, if any, was achieved with the support of this technical assistance?	Optional	Open Response	Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Evaluation Plan Outcome: Lead Agencies report progress toward reaching their goals with the assistance of NCSIA Large-scale TA Projects: ASSIST – Application of skills or concepts from ASSIST TA used in participants’ work
	The TA increased your agency’s access to examples of best practices and innovations implemented by other CCDF grantees.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Targeted, Intensive	Desired Results: #4 Opportunities to learn from others Large-scale TA Projects: CBC Landscape Scan – Question in PWS
Increased Knowledge and Skill	How much did the event increase your knowledge of (specific topics) presented?	Recommended	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Large-scale TA Projects: ASSIST – Increase in knowledge of the topic(s) about which ASSIST participants have requested TA PICCC – Increase in knowledge of this series and the topics presented in the modules Cost-based Subsidy Payments - Increase in Lead Agency staff knowledge on gathering, analyzing, and using cost data to inform subsidy rates; Increase in Lead Agency staff’s knowledge of OCC requirements, guidance, and best practices on cost-based alternative methodology
	How much did the event increase your comfort with communicating about (specific topic(s) presented)?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Large-scale TA Projects: PICCC – Increase comfort in presenting the train-the-trainer series to front-line staff
	How much did the technical assistance/event/session increase your skill level in planning and implementing related policies and practices?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Evaluation Plan Outcome Lead Agencies report progress toward reaching their goals with the assistance of NCSIA Large-scale TA Projects: ASSIST – Increase in participant skill level for assessing, planning, and implementing policies and practices related to the TA they received
	How much did the technical assistance increase your agency’s awareness and use of high-quality training or technical assistance opportunities?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Individualized, Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, effective and clear TA Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies Large-scale TA Projects: PICCC – Increase in knowledge of this series and the topics presented in the modules; Increase comfort in presenting the train-the-trainer series CBC Landscape Scan – Question in PWS
	How much did the technical assistance increase your agency’s access to and use of training or technical assistance resources and tools designed to support (specific topic area) staff orientation and ongoing training?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, effective and clear TA Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies Large-scale TA Projects: PICCC – Increase comfort in presenting the train-the-trainer series to front-line staff
	How much did the technical assistance increase your agency’s compliance with CCDF rules and policies?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, effective and clear TA Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies Lead Agencies report progress toward reaching their goals with the assistance of NCSIA Large-scale TA Projects: ASSIST – Increase in participant skill level for assessing, planning, and implementing policies and practices related to the TA they received CBC Landscape Scan – Question in PWS
	How much did the technical assistance increase your agency’s use of CQI techniques?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, effective and clear TA Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies Large-scale TA Projects: ASSIST – Increase in participant skill level for assessing, planning, and implementing policies and practices related to the TA they received CBC Landscape Scan – Question in PWS
	How much did the technical assistance increase integration of your agency’s functions (e.g., administrative, technological, policy)?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, effective and clear TA Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies Lead Agencies report progress toward reaching their goals with the assistance of NCSIA Large-scale TA Projects: CBC Landscape Scan – Question in PWS
	How much did the technical assistance increase your Tribal agency’s access to TA support in establishing agreements with states?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Targeted, Intensive	Desired Results: #3-4 – Effective and clear TA, and opportunities to learn from others Evaluation Plan Outcomes: Lead Agencies report progress toward reaching their goals with the assistance of NCSIA Large-scale TA Projects: CBC Landscape Scan – Question in PWS Large-scale TA Project:
	How much did the technical assistance increase your agency’s ability to establish and leverage partnerships to improve coordination and collaboration with other agencies?	Optional	Scaled item No Increase (1) Small Increase (2) Moderate Increase (3) Large Increase (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #4 – Opportunities to learn from others Evaluation Plan Outcomes: Lead Agencies report progress toward reaching their goals with the assistance of NCSIA Large-scale TA Projects: CBC Landscape Scan – Question in PWS
	BEFORE this technical assistance, my knowledge of (specific content/topics addressed) can best be described as … AFTER this technical assistance, my knowledge of (specific content/topics addressed) can best be described as …	Optional	Scaled item No knowledge (1) Minimal knowledge (2) Moderate knowledge (3) High level of knowledge (4)	Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Large-scale TA Projects: ASSIST – Increase in knowledge of the topic(s) about which ASSIST participants have requested TA PICCC – Increase in knowledge of this series and the topics presented in the modules Cost-based Subsidy Payments - Increase in Lead Agency staff knowledge on gathering, analyzing, and using cost data to inform subsidy rates; Increase in Lead Agency staff’s knowledge of OCC requirements, guidance, and best practices on cost-based alternative methodology
	BEFORE this technical assistance, my comfort with communicating about (specific content/topics addressed) can best be described as … AFTER this technical assistance, my comfort level with communicating about (specific content/topics addressed) can best be described as …	Optional	Scaled item No comfort (1) Minimal comfort (2) Moderate comfort (3) High level of comfort (4)	Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Large-scale TA Projects: PICCC – Increase in knowledge of this series and the topics presented in the modules
	The TA provided increased your agency’s use of audits to identify and address policy and procedural gaps.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Evaluation Plan Outcomes: Lead Agencies report progress toward reaching their goals with the assistance of NCSIA Large-scale TA Projects: CBC Landscape Scan – Question in PWS
	The TA increased your agency’s use of process mapping and/or systems analysis to identify and address policy and procedural gaps.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Evaluation Plan Outcomes: Lead Agencies report progress toward reaching their goals with the assistance of NCSIA Large-scale TA Projects: CBC Landscape Scan – Question in PWS
Intended Application of Knowledge	I learned something during this event that I plan to use in my work.	Recommended	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Evaluation Plan Outcomes: Lead Agencies report progress toward reaching their goals with the assistance of NCSIA
	BEFORE this technical assistance, my comfort with implementation efforts related to (specific content/topics addressed) can best be described as … AFTER this technical assistance, my comfort level with implementation efforts related to (specific content/topics addressed) can best be described as …	Optional	Scaled item No comfort (1) Minimal comfort (2) Moderate comfort (3) High level of comfort (4)	Individualized, Targeted, Intensive	Desired Results: #3 Effective and clear TA Evaluation Plan Outcomes: Lead Agencies report progress toward reaching their goals with the assistance of NCSIA Large-scale TA Project: CBC Landscape Scan – Question in PWS

Optional Constructs: The following four constructs and related questions are optional, designed for planning future technical assistance, providing internal feedback to presenters, and improving technical assistance delivery. These constructs are Satisfaction, Presenter Quality, Formative Assessment, and Barriers to Application of Knowledge.

Construct	Question	Type	Responses	Type of TA for Use of Question	Alignment with NCSIA Logic Model
Satisfaction	I was satisfied with the quality of this technical assistance/session/event.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #1 – Trusted partner
	I was satisfied with the timeline of this technical assistance/session/event.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Targeted, Intensive	Desired Results: #1 – Trusted partner
	I would recommend this technical assistance/session/event to my colleagues	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Targeted, Intensive	Desired Results: #1 – Trusted partner
	What is your preferred way of learning virtually?	Optional	Choices Small Group Discussions Large Group Discussions Presentations Working Session Combination of Above	Universal, Individualized, Targeted, Intensive	Desired Results: #2 – Differentiated support Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies
	What is your preferred technical assistance session length?	Optional	Choices Less than 60 minutes 60 minutes 90 minutes More than 90 minutes	Universal, Individualized, Targeted, Intensive	Desired Results: #2 – Differentiated support Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies
	What is your preferred meeting schedule for ongoing technical assistance?	Optional	Choices Every Two Weeks Once per Month Quarterly Other _____	Universal, Individualized, Targeted, Intensive	Desired Results: #2 – Differentiated support Evaluation Plan Outcomes: NCSIA TA is tailored when possible, to the individual needs of Lead Agencies
Presenter Quality	The presenters were knowledgeable in the content area.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #1 – Trusted partner
	The presenters were effective in engaging participants.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #1 & 3 – Trusted partner and effective and clear TA
	The presenters were responsive to participants' questions.	Optional	Scaled item Strongly Disagree (1) Disagree (2) Agree (3) Strongly Agree (4)		Desired Results: #1 & 3 – Trusted partner and effective and clear TA
Formative Assessment	How can we improve this technical assistance/session/event?	Optional	Open response	Individualized, Targeted, Intensive
	What topics would you like to learn more about in the future?	Optional	Open response	Universal, Individualized, Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, and effective and clear TA
	What type(s) of follow-up support or resource(s) would be most useful to you on (this specific topic?	Optional	Open response	Universal, Individualized, Targeted, Intensive	Desired Results: #3 – Effective and clear TA
	What is the likelihood you or your staff would participate in a peer learning opportunity on this topic?	Optional	Scaled item Very unlikely (1) Unlikely (2) Likely (3) Very likely (4)	Universal, Individualized, Targeted, Intensive	Desired Results: #1 & 3 – Trusted partner and effective and clear TA
	What additional T/TA opportunities would help you further improve your policies or practices?	Optional	Open response	Individualized, Targeted, Intensive	Desired Results: #3 – Effective and clear TA Large-scale TA Projects: ASSIST - Increase in participant skill level for assessing, planning, and implementing policies and practices related to the TA they received
	Would you be willing to share the work of your Lead Agency related to this technical assistance opportunity?	Optional	Yes (1) No (2)	Universal, Individualized, Targeted, Intensive	Desired Results: #4 – Opportunities to learn from others
	How did you find out about this event?	Optional	Open response	Universal, Individualized, Targeted, Intensive	Desired Results: #4 – Opportunities to learn from others
	What are you most interested in learning about (this specific topic)?	Optional	Open response	Universal, Individualized, Targeted, Intensive	Desired Results: #2 – Differentiated support
Barriers to Application of Knowledge	What factors, if any, may prevent you from using what you learned?	Optional	Open response	Individualized, Targeted, Intensive	Desired Results: #1-3 – Trusted partner, differentiated support, effective and clear TA