Video Documentation Online Exhibit Evaluation Online Journal History Private Lessons Living Loess

Evaluation using Qualitative Methods

In this tab I lay out a path that I took to become a documenter and evaluator. This is the most frequent question that I am asked about the work that I do. I did not follow a career path, because this work did not exist. I moved step by step to solve problems of reporting and describing events as new technologies arrived that added new dimension to this work. For this reason, I am including early work, so that you will be able to see how we used these technologies when they were brand new, and still not entirely user friendly or elegant in design.

My work in evaluation began in the early 1990's when I was invited to use field study methods, photography and video to create documents that described innovative programs in urban and suburban schools and communities. The programs had complex designs and combined new teaching skills and materials, and technology. Funders wanted a clear picture of the kinds of experiences the students and teachers were having. For me, it was a perfect integration of field research and video/photography that could be assembled into multimedia documents. At the same time, the Internet was opened, and I saw the potential for using hyperlink formats to distribute these documents. Here are examples of some of my documentation and evaluation work.

Parents and teachers in a suburban community were trained as docents by the Nature Conservancy to accompany children from the school district into natural areas of their village park district. The format was expanded to include field trips to natural areas in the Cook County Forest Preserve District. In addition to field study, children did stewardship such as pulling weeds or collecting seeds of native plants. Teachers and parents from the suburban school paired with teachers and parents from a Chicago Public School as part of a sister schools project. They teamed up to train parents from the urban school as docents, and both schools shared an environmental stewardship project in a natural area in the Forest Preserve District.

What my documentation made clear was the way children and parents both engaged in this work, and how it supported classroom learning using integrated activities. For example, students used math skills to measure and draw maps of natural areas, and writing skills to write up field journals and site observations based on the work. They also worked with their classroom teachers to write Haiku poems expressing their feelings about field study, and being land stewards. This pilot project culminated in The Mighty Acorns program of The Illinois Nature Conservancy. It is a school based youth stewardship program with trained community docents and teachers. Parents and teachers used this documentation in a presentation they made at the Midwest Oak Savanna Conference in February, 1993.

Teachers began to organize their own field experiences in natural areas. Here is a write up of a field study day they organized at the Illinois Dunes. They developed the outing with the art teacher. Later, she demonstrated how to sketch dunes. There was an obvious difference between the drawings of students who visited the dunes and students from other classes who had not. Student Pastel Drawings of Dunes (These archival files will display better if you pull in the right edge of the browser - click and drag the lower right corner) Students later reported their data from measurement instruments, and wrote up summaries of what they learned about natural areas. Student Essays with photos by Katy Beck (Fifth grade teacher) We were not using digital cameras yet, so my field notes are illustrated from slides, and Katy's are from photo prints. We digitized photos with a scanner. We limited the size of photos to 25 kb in those days. It took a long time for a photo to fill in on a page, so we kept the images at low resolution and small in height and width. These web pages were created on Page Mill, an html editor by Adobe. Later I learned to write my own coding to gain more control over the way text and image are displayed.

In addition to creating an archive of the trip, I included topics for reflection. This one focuses on the tension between a tour, and open ended exploration of an area.Spontaneous Activities

We were interested in bringing education beyond classroom walls, into field settings. Field training for teachers went along with field study for students. Here is a pdf write up of field work and interviews I did with Chicago Public School teachers who combined a botany class with field study in a park near their school. Elfriede Pergams, offered a Botany and Horticulture workshop as part of the ongoing workshops for teachers at the Chicago Teachers' Center. She had a strong science background and many years of teaching experience at CPS. I videotaped the workshop, field study, and teacher interviews. I edited a video which was used at the Chicago Teachers' Center and by the Forest Preserve District of Cook County. The images for the pdf are video freeze frames, and the text is transcribed from the video. I am going into all of this here as an example of the transition we were making to use technology for field based research.

I taught video documentation workshops to teachers at my video studio. We started at North Park Nature Center, working as a team to create a video archive of ecological restoration activities. In my studio we studied and logged the archive, and collaborated to edit a short video. It was an intense weekend that gave teachers a new perspective on video documentation. Each wrote a description of how the experience would carry forward into their teaching practice. Classroom Applications They began to see how technology might connect their students to other students through joint projects, and how students might begin to use video. We were fascinated by the properties of video for education. We saw video footage as a point of view of record, that could be framed through editing, and serve as a rich memory of sound, image and movement. It could be saved in an archive and used again and again in new formats of expression. It was raw material that could be gathered, shared, and exchanged. We imagined that our students would be co-creators of huge archives of experience that could be downloaded and combined with other pieces of experience without limits. We imagined that their points of view would be included with those of their whole generation, and through this process they would learn about one another. We saw ourselves sitting back to see what they would do, and how it would shape who they would become.

The Chicago Teachers' Center at Northeastern Illinois University invited me to video document their work in arts integration as part of the Chicago Arts Partnerships in Education. This was the start of 18 years documenting and evaluating arts integration programs with their CAPE, LEAP, LEAP-Annenberg, Arts at the Center, and Every Art Every Child projects. My assignments gradually morphed from documentation to evaluation, as the requirements of funders shifted from wanting descriptive documents to qualitative data summaries and scores. I began to quantify qualitative data as measurable summaries of teaching practice and children's work. For the last 10 years, I have focused on
          Collecting qualitative data
          Using it to create instruments to score these data
          Training teachers to use video and photography to create their own qualitative records
          Conducting group and one-on-one interviews with teachers and artists participating in arts integration
          Converting student writing and art based on Studio Habits of Mind into summaries of the
               learning process, from children's perspectives.
It was funded with Professional Development for Arts Education (PDAE), and Arts in Education Model Development and Dissemination (AEMDD) Arts in Education grants from the US Department of Education. These were conducted in the Chicago, Berwyn,and Elgin Illinois public school districts.

In the late 1990's, Dr. Elizabeth Landerholm, coordinator of the Early Childhood Education program at Northeastern invited me to evaluate an Even Start after school literacy program. We set up an evaluation team each year to photo and video document Even Start from multiple perspectives (parent assistants, parent technicians, children, teachers, interns, school administrators, graduate and undergraduate university students, student teachers, professors, and group interviews.) We assembled a multi-media book every year for all 8 years of the Even Start grant. This documentation was aligned with quantitative data summaries in annual reports to the US Department of Education. When the ECE program needed to renew its NAEYC accreditation, these documents were used as evidence of clinical and community experience.

I continued to work with Dr. Landerholm as evaluator of a pre-school literacy grant with the Chicago Teachers' Center and the Children's Center of Cicero-Berwyn. In addition to using the ELLCO literacy instrument, and other literacy tests assessments for pre-school children, I systematically collected field data and photographs of classroom practice, classroom organization and materials. These data were aligned with student and classroom scores in annual reports in Early Reading First. Teachers and administrators at CCCB were trained in documentation and photography. At the end of the grant they were able to apply this training to Online Creative Curriculum reporting that uses field data and photographs.


Return to Menu