Faculty Comments about AE Design Teaching and the Survey

In other blog entries and the AEEE paper I've summarized the results of the survey.

The last two questions in the survey were open-ended, asking faculty for their comments about both the overall subject of the survey and the questionnaire itself.  Fifty one (out of 105) faculty replied to one or the other of the questions - including the survey author.  All the comments are on the following web page as well as the survey text itself in PDF form.  They are edited only to remove identifying information.

• Faculty Responses to the Final Questions on Survey
• The Survey questions themselves in PDF format

Defining AE Design - ASEE-2009 Paper (AC 2009-448)

Next week I'll be giving a paper based on the survey that 105 faculty in the 17 schools that I visited completed.  The paper in the ASEE proceedings was completed before all the survey results were collected.  This version (link) provides updated results.  There were some changes in both the coding methodology and the results between the two that are noted in the paper.  This one is more complete and appropriate.
The abstract is:

---

An online survey of representative faculty at ABET-accredited Architectural Engineering schools addressed the question of what constitutes “Architectural Engineering Design” (AED). The faculty are first characterized in multiple ways: university, academic rank, years of experience, registration status and discipline. The results of their open-ended definition of AED are examined using nine categories derived from responses rated on 1-5 Likert scales, with the analysis broken down using the same faculty characterization. Faculty opinions about the disciplines necessary to include in AED are also analyzed. Overall there is general agreement that disciplinary “skills” are an important part of AED as are, to a lesser extent, the “products” produced. There is some agreement about the idea of “integration” of the disciplines and much less agreement on many of the other concepts, with several barely mentioned. Most faculty feel that their definition of AED is the same as their school’s, but many express uncertainty about the existence of a national definition. Similarly there is considerable agreement that more than one discipline (Architecture, Structure, HVAC, Electrical, Construction Management) is required to constitute AED, but there is marked disagreement about what specific ones should be included, with opinions ranging from two to all five.

PDF of Talk Slides

Evaluation of Issues and Need for Research on Teaching

Respondents were asked to give ratings (1-5 scale using words) on the following questions:

• Evaluate the importance of the following issues to your school
• Evaluate the importance of the following issues to AE programs nationally
• Evaluate the importance of research related to teaching AE Design in the following areas

The detailed results are available on this Google-Docs spreadsheet.

Summary of Results

Issues for Schools

Considerable Importance

• Integration of all three ABET components of AE - Structure; MEP; Construction Management (3.9/5)
• Sustainability Design (3.5/5)

Non-Significant Evaluations

• All the others were not rejected from the null-hypothesis of "average" significance.

Issues Nationally

Considerable Importance

• Integration of all three ABET components of AE - Structure; MEP; Construction Management (3.6/5)
• Sustainability Design (3.7/5)

Some Significance

• BIM (Building Information Modeling)  (3.4/5)
• Lean/Economic Design (3.3/5)

Non-Significant Evaluations

• All the others were not rejected from the null-hypothesis of "average" significance.

Teaching Research

Considerable Importance

• Correlating industry needs with AED curricula  (3.8/5)
• Identifying the most effective teaching strategies for different aspects of AED. (3.5/5)
• Incorporating distance-learning into the curriculum (2.5/5) - [negative importance]

Some Significance

• Incorporating computer-based instruction into the curriculum  (3.4/5)
• Addressing different student learning styles (3.4/5)

Non-Significant Evaluations

• All the others were not rejected from the null-hypothesis of "average" significance.

Note on Statistical Significance of Results

Almost all the results showed a statistically significant difference from a null-hypothesis model centered on the middle term (3.0) of the 5-Likert values using a goodness-of-fit chi-squared analysis.  Those that weren't significant are shaded gray. 

Note, as always, that a significant result does not mean a big difference.  That is a matter of individual judgement, which is the reason the actual counts are shown.

AE Schools Cooperation & ABET Issues

Respondents were asked to give ratings (1-5 scale using words) on the following questions:

• Evaluate the importance of cooperation and coordination between the different AE Schools in the US (or abroad).
• Evaluate the importance of possible changes in the ABET accreditation requirements.

The detailed results are available on this Google-Docs spreadsheet.

 

Summary of Results

Cooperation and coordination between the different AE Schools in the US

Considerable Importance

• ABET criteria development (4.0/5)
• PE Exam Development (3.7/5)

Some Importance

• Sharing Instructional Techniques (3.3/5)

Neutral or Negative Importance

• Remote access to specialist instructors (3.1/5)
• Joint projects with other schools (2.8/5)

 

Importance of Possible Changes in the ABET accreditation requirements

Considerable Importance

• Require "integration" of at least two of the three AE components (3.6/5)
• Require industry experience for faculty members teaching AE Design. (3.4/5)

Some Importance

• Identify a specific number of credits identifiable as AED (3.2/5)

 

J. Mitchell Suggestion

While there isn't unanimity on these results, several show a considerable weight that deserves discussion in the AEI academic council and at the AEI national level.

 

Note on Statistical Significance of Results

Almost all the results showed a statistically significant difference from a null-hypothesis model centered on the middle term (3.0) of the 5-Likert values using a goodness-of-fit chi-squared analysis.  Those that weren't significant are shaded gray. 

Note, as always, that a significant result does not mean a big difference.  That is a matter of individual judgement, which is the reason the actual counts are shown.

Teaching Approaches Used and Advocated

Respondents were asked to give ratings (1-5 scale using words) on the following questions:

• I use the following approaches in teaching AE Design
• I use student teamwork in teaching AE Design.
• I would advocate changing the way the school teaches AED by changing the use of the following techniques

The detailed results are available on this Google-Docs spreadsheet.

 

Summary of Results

Approaches used:

• Lecture is most common, followed by problem-based learning and guided design.
• Architectural studio and modified study are never used in many cases.

Student Teamwork Used:

• There is quite a spread in use in most cases - no consensus
• Teamwork is used not much more than "sometimes" in most approaches, with a fair amount of use in Problem-Based Learning, and Guided Design.

Advocating Change in use of Techniques

• There is considerable interest in increasing the use of all techniques except Lecture.
• There is a marked interest in decreasing the use of lecture

 

Note on Statistical Significance of Results

Almost all the results showed a statistically significant difference from a null-hypothesis model centered on the middle term (3.0) of the 5-Likert values using a goodness-of-fit chi-squared analysis.  Those that weren't significant are shaded gray. 

Note, as always, that a significant result does not mean a big difference.  That is a matter of individual judgement, which is the reason the actual counts are shown.

Definitions of Architectural Engineering Design

The  Google-docs spreadsheet linked here presents the first batch of data from which I'll draw conclusions, starting with the paper I'll be giving at the ASEE Conference in June (6/16 2:15-4:00 - #2506: Beneficial Case Studies in AEC Education). 

It should be interesting to those who want to know what are the definitions of AE Design given by the faculty, and also what are the characteristics of the AE faculty.

What you'll find in the spreadsheet are:

AE Design Definitions - Gives all the definitions of AE Design associated by the University of the respondent. It also gives their comments if they felt that their definition was different either from their school's definition or the national definition. Finally, it gives the coding on 9 scales I developed.

Characteristics - Gives the summary of characteristics of the respondents and their mean scores on the coding of the definition of AE Design.

Interviews by Univ - Gives the number of oral interviews conducted at each university during my sabbatical visits.

Anonymity

I believe I've been very careful to remove any identifying information either directly or through cross reference, while attempting to leave the definitions as the respondents entered them as intact as possible.

If you find any means to identify yourself or another individual please let me know so I can correct the problem.

Survey Now Closed

The survey of faculty closed this morning, having gained about twenty responses since my mid-April email to faculty who I hadn't recorded as having already completed the survey.

I've begun analysis of the data and shortly will post the first piece results - of the definitions of Architectural Engineering Design (AED) and the characteristics of the respondents.

There were 102 respondents, of whom 87 gave a definition of AED.

I interviewed 137 people during my sabbatical visits.  Assuming that the overlap is high, that means that the response rate to my survey is about 74% overall, a very good response rate, particularly for a survey as lengthy as the one I presented.  Thanks to everyone.