Theoretically and Empirically Based Evaluation of Laboratory Courses – PraQ Questionnaire

Abstract

Abstract (1) Purpose: Since the Bologna Process was introduced, a claim for permanent screening of learning outcomes at higher education level has started (Friedrich, 2005; Hopbach, 2007). Nevertheless, evaluation is mostly not based on profound theoretical background, and seldom accepted by participants (Csonka et al., 2014). (2) Academic significance: Two theoretically grounded psychological test instruments, called LeKo (Thiel et al., 2012) and BeVaKomp (Braun et al., 2008), were created at FU Berlin in order to change this deficit. They were successfully applied at universities, especially for lectures and seminars, and now replace the standard evaluation form university wide. Unfortunately, they ignored science laboratories for their theoretically based and comprehensive research. Lab courses are the core of experimental training in scientific education (Psillos & Niedderer, 2002). Keeping this in mind, the theory based evaluation of those courses would clearly fill a gap. They aim to teach experimental practices (e.g. Schreiber, Theyßen, & Schecker, 2012) – an essential part of further studies and the future work of the students. Therefore, our goal was to construct a theoretical framework and an economically usable questionnaire. PraKo questionnaire was constructed in 2014 and piloted in 2015 (german “Praktikums-Kompetenzen”: science lab practices). (3) Theoretical framework: The PraKo questionaire is based on a theoretical model for lab quality (Rehfeldt, Mühlenbruch, & Nordmeier, 2015), containing three main dimensions: (a) learning gains (growth of competencies), (b) teaching practices of tutor, (c) lab material. (4) Method: The measuring tool itself consists of 140 items in 40 scales, divided into two questionnaires (15min each). The first part measures learning gains (a) regarding growth in content knowledge, scientific inquiry practices, communication etc. The second part contains the following: the teaching practices of lab tutors (b) measuring abilities like explaining properly, summarizing or emphasizing relevance. Furthermore it contains the material-dimension (c) which covers the lab script, integration of the lecture, and basic experimental material (Fraser & McRobbie, 1995; Kreiten, 2012). The analysis of the piloting data demands an exploratory factor analysis for the new and highly modified scales, well established scales are to be integrated in CFAs. (5) Data sources: The data sources for piloting the PraKo consist of several science labs across Germany, including different disciplines (physics, chemistry, biotechnology etc.) and different universities (Berlin, Potsdam, Wildau, Kiel, Bielefeld, Wuppertal, Tübingen, Cologne und Vienna), resulting in N = 329 for the learning gain dimension and N = 242 for teaching practices and material. Extending investigations is planned. (6) Results: Results for the factor structure of the inductive scales concerning teaching practices and material dimension show a 8 factor solution very close to the intended structure, having good reliability among the scales. Therefore, PraKo questionnaire can be used for research on labs from now on, starting with a validation study for further data.