A successful student-initiated assessment method for an environmental engineering graduate program
Journal
ASEE Annual Conference and Exposition, Conference Proceedings
Date Issued
2006
Author(s)
Abstract
Obstacles in assessing academic conditions can include generating interest in assessment efforts in order to achieve high response rates, transcending communication barriers, preserving confidentiality, minimizing biases from numerous sources, and conducting meaningful statistical analyses. A graduate environmental engineering program needed to overcome these obstacles to create a valid assessment tool. Previous program surveys did not amply address specific student concerns. Those surveys had poorly designed questions and answer formats. Survey distribution had relied on students to retrieve and return surveys themselves. Data analysis had consisted of only computing mean values and compiling comments. Results of the surveys had suffered from low response rates, biases, and demographic underrepresentation. A graduate-student committee designed a survey considering the aforementioned problems. "The improvement of research quality" was the overall survey theme, and four subtopics research resources, research preparation, research views and attitudes, and research-group support - were created to generate specific question ideas from the student population at-large. Questions were included in the survey based on importance, the actionable nature of obtained knowledge, and other criteria. Background and control questions were included for categorizing respondents. Sensitive natures of some questions were addressed to reduce biases. The format of the survey was tailored to make respondents comfortable and interested in participating. Question quality was examined through a pilot study and reviews by professionals. Answer formats were mainly closed-ended with most open-ended questions providing supplemental information. Hand-distribution and hand-collection were intended to make the survey tangible, appreciable, and accessible for respondents. Univariate analysis produced meaningful findings regarding individual variables, while bivariate/multivariate analysis determined correlations among multiple variables. Sensitivity analysis was also conducted to uncover potential biases in answering behavior for students who both were involved in survey design and responded to the survey. We submit that our survey effort was successful overall due to high response, accurate demographic representation, positive student feedback, reduced biases, and significance of findings. 50 students (greater than 75% of the population) responded to the survey. Some of the salient findings indicate deficiencies in communication and statistics education, deficiencies in overall research preparation for first-year and master's students, an overall failing of a laboratory course to provide research-skill education, and a lack of guidance from research-group members for some students. Our improved survey has led students and faculty members in the program to appreciate internal assessment and encourage the student committee to continue its efforts. The committee is beginning to solve problems discovered in this study and will be continuing to use the successful method in further assessment. Our method is believed to be applicable for engineering programs that must deal with common obstacles in making a sound assessment tool. © American Society for Engineering Education, 2006.
Other Subjects
Data reduction; Problem solving; Statistical methods; Students; Assessment method; Communication barriers; Engineering programs; Multivariate analysis; Environmental engineering
Type
conference paper