How To Conduct An Internal Assessment Using Parenting Interactions with Children: Checklist of Observations Linked to Outcomes (PICCOLO™)

*The PICCOLO was originally designed to evaluate parenting behaviors with children; however, emerging research indicates it is also appropriate to use to evaluate teacher/provider interactions with children. For PA Keystone STARS, the PICCOLO is used to observe teacher/provider interactions with children.

NOTES: Although the PICCOLO can be used for internal assessment, it can also be conducted as an external assessment by a PICCOLO trained and reliable Pennsylvania Key (PA Key) Program Quality Assessor (hereinafter referred to as “Assessor”).

When using the PICCOLO for an internal assessment to meet the expectations for one of the Keystone STARS Standards, please consult the most current version of the Keystone STARS Performance Standards and the Keystone STARS Program Manual.

How to conduct an objective, authentic classroom assessment 

The Keystone Stars Standards expect programs to conduct internal assessments annually and, once every three years, engage their internal assessment team in discussion of their internal assessment results with an Assessor from the PA Key. 

• The internal assessment should be an observation; therefore, someone who has knowledge of the selected Program Observation Instrument (POI)and its expectations should plan to observe for the recommended amount of time (for the specific POI), in the classroom.  
• Keep in mind, the internal assessment is not something to be given to the classroom teacher to complete as a checklist of things that the teacher feels occur in the classroom. Teachers cannot observe themselves in action and may not provide an objective picture of their classroom. The observer should be able to commit to observing and taking notes and should not have any classroom responsibilities during the observation. 
• Family child care providers, working alone, should discuss observation options with the Assessor or Quality Coach. 
• The observation should occur when most of the children are present and actively engaged in the classroom so interactions with peers, teachers, and materials can be observed; therefore, no part of the observation should occur during naptime unless naptime is a required component of the POI. 
• The internal assessment should be viewed as an integral part of a program’s commitment to continuous quality improvement rather than just another piece of paperwork to be completed to meet an expectation. Information from an objective internal assessment can yield much information about the program’s practices, providing a foundation for the program to celebrate their successes and build upon existing practices. 

Assessment at-a-glance

• One hour to complete a classroom observation
• Ages 3 months to 73 months (6 years one month)
• Assessment is based on observation only
• Uses a 3-point scale
• Focuses on domains: Affection, Responsiveness, Encouragement, and Teaching
• Available in Spanish (found in the PICCOLO User’s Guide)

Required materials

• Parenting Interactions with Children: Checklist of Observations Linked to Outcomes User’s Guide (PICCOLO) ISBN-13 978-1-59857-302-2 or ISBN-10-1-59857-302-0
• PICCOLO score sheets ISBN-13-978-59587-303-9 or ISBN-10-59857-303-9

Supplemental materials

• The PICCOLO Training DVD ISBN 978-1-59857-304-6

Purchase the materials

• There may be multiple sources where the required and supplemental materials can be purchased. Searching online by resource name or ISBN may yield several options.
• Be sure to allow adequate time for ordering and delivery of all resources.
• Score sheets cannot be duplicated and must be purchased.

Conduct the internal assessment (start to finish)

1. Learn about PICCOLO

• Register for the online self-paced Foundations of Parenting Interactions with Children: Checklist of Observations Linked to Outcomes (PICCOLO) course on the Professional Development (PD) Registry (papdregistry.org) (available sometime in 2024/2025).
• Request a Facilitated Program Observation Instrument Orientation (FPO). This one-hour session with an Assessor provides time to learn basic information about the PICCOLO and an opportunity to ask specific questions. The FPO is structured to meet each individual’s or program’s specific needs, and each person attending receives one hour of PQAS credit. Please note the FPO is not a substitute for the online self-paced PA Foundations of Parenting Interactions with Children: Checklist of Observations Linked to Outcomes (PICCOLO) course, but rather provides a basic overview.
• The more you learn about the quality practices in each PICCOLO subscale, the more accurate your internal assessment will be.
• It is recommended that the administrators and internal assessment team become familiar with the expectations of the PICCOLO prior to the internal assessment.
• Access informal professional development and learning options for the PICCOLO, including articles, videos, and other resources on the PA Keys website.
• Read the PICCOLO User’s Guide to learn the basic information about the PICCOLO and guidance on coding practices. Additional information is available on the PICCOLO Training DVD: Implementation and Scoring.
• Watch video clips of parent-child interactions on the PICCOLO Training DVD. The videos offer an opportunity to score PICCOLO behaviors and compare your scores to the authors for those vignettes, which are provided in Appendix C of the PICCOLO User’s Guide.
• Practice observing staff interactions. If you are comfortable with it and it is an approved program practice, video a staff person and child together for internal training. Practice scoring the video by watching it with someone else. Score the PICCOLO domains separately and then compare your ratings. Watch the video again and discuss the differences in your ratings until you can come to an agreement for all items.
• Visit the Brookes Publishing website to find other valuable PICCOLO resources.
• Clarify any questions with an Assessor.

3. Prepare to conduct the assessment

• If time has elapsed since your purchase and review of the PICCOLO, re-familiarize yourself with the scale expectations.
• Gather the materials needed to conduct the observation, including a digital device for notetaking (if preferred), paper/pencil, a copy of the PICCOLO User’s Guide, and a PICCOLO score sheet.
• Set aside at least one hour to conduct the assessment.
• Plan to complete the assessment in one sitting.

4. When to conduct the assessment

• When you have at least one hour of uninterrupted time
• When you are free from caregiving or administrative responsibilities
• A variety of activities with varying materials can be observed. Choose activities that:
  • Fit program goals.
  • Can take place in a small space.
  • Engage both the adult and the child.
• It may be helpful to tell the children that a guest will be visiting their classroom for a special project. They will watch them play and do their work but will not be able to play with them while doing their work.

5. Conduct the internal assessment

• Do not try to score while observing; complete the score immediately after the assessment based on your notes.
• Spend approximately one hour observing.
• Videotaping the observation for your internal use only is highly recommended if permitted by your program.
• Be objective. It might be second nature to give scores based on what you observe your teachers/providers do regularly. The PICCOLO is designed to look at quality in a specific amount of time. If the practice is observed during your scheduled assessment time, count it; if it is not observed during that time, do not count it.
• Read every indicator of quality carefully. For specific examples of items, refer to the PICCOLO User’s Guide.
• Read all supplemental information. Review the Observation Notes for the four domains in the PICCOLO User’s Guide.

6. Score the assessment

• If scoring a live observation, score immediately after the observation is completed. This is when the observed practices are easiest to recall.
• If scoring a video observation, score after reviewing the video.
• Score all four domains.
• The 29 items are scored on a three-point scale:
• 0-Absent: no behavior observed
• 1-Barely: brief, minor, or emerging behavior
• 2-Clearly: definite, strong, or frequent behavior
• Add the item scores in each domain to get a domain score; add the domain scores to get a total PICCOLO score.
• Additional scoring information can be found in the PICCOLO User’s Guide, page 24

7. Review assessment findings

• Conducting a PICCOLO assessment is only the first step toward growth. Review your assessment results with your Internal Assessment Team, Quality Coach, and Assessor, and brainstorm areas for Continuous Quality Improvement (CQI) goals.
• The best way to interpret the PICCOLO scores is by looking at high domain scores or high item scores as strengths. All staff do some positive things when interacting with the children.
• It is important to celebrate your strengths. Improvements often build upon strengths.
• Talk with the staff about their perceptions of observed interactions with the child(ren).

Next steps

• Assure everyone has contributed ideas and ensure everyone knows the next steps to take toward improvement.
• When planning for CQI goals, look toward implementing the next level in the indicators. Incorporate your goals into your CQI plan.
• Start small. Focus on one or two indicators at a time.
• Be patient. Do not be discouraged if your initial scores do not meet your expectations. Growth takes time.
• Change what you can and accept things outside your program’s ability to improve.
• Share your growth/CQI plan with everyone on your team.
• Plan to regularly revisit your goals and program’s administrative practices to assess and update as goals are achieved. The CQI process is cyclical as you continuously reflect on your practices and drive for growth.