Computer Vision for the Built Environment

Course Information

• Course ID: CEE 247C (Winter 2024)
• Instructors: Prof. Iro Armeni
• Teaching Assistants: TBD
• Lectures: Wednesdays 1:30 - 4:20 PM in TBD
• Credits: 3 or credit/no credit
• Online Material: Canvas

Course Description

Summary

The course is an introduction to Visual Machine Perception technology – and specifically Computer Vision and Machine Learning (CV-ML) – for the built environment. It will explore fundamentals in this technology both in research and products, in tight reference to design, construction, and operation/management. It will consider the current and potential impact of this technology on achieving sustainability goals, such as related to reuse, circularity, and performance-based lifecycle, as well as the organizational considerations behind development and adoption.

About

The past few years a lot of discussion has been sparked in AEC on CV-ML for the built environment. Despite advancements in this interdisciplinary field, we still have not answered fundamental questions about adopting and adapting CV-ML technology. In order to achieve this, we need to be equipped with rudimentary knowledge of how this technology works and what are essential points to consider when applying CV-ML to this specific domain.

In addition, the availability of sensors that collect visual data in commodity hardware (e.g., mobile phone and tablet), is creating an even bigger pressure in identifying ways that new technology can be leveraged to increase efficiency and decrease risk in this trillion-dollar industry. However, cautious and well-thought steps need to be taken in the right direction, in order for such technologies to thrive in an industry that showcases inertia in technological adoption and to drive sustainability goals.

The course will unfold in two interwoven storylines:

• The first storyline will introduce fundamentals in computer vision and machine learning technology, as building blocks that one should consider when developing related applications. These blocks will be discussed with respect to latest developments (e.g., deep neural networks), pointing out their impact in the final solution.

• The second storyline consists of AEC processes, namely architectural design, construction, and operation/management. These processes will serve as application examples of the technological storyline.

Students will see the importance of taking into account the application requirements when designing a CV-ML system, as well as their impact on the building blocks. Guest speakers from both the CV-ML and AEC domains will complement the lectures.

Learning Goals

By the end of the course students will develop computational thinking related to visual machine perception applications for the built environment and the architecture, engineering, and construction (AEC) domain. Specifically, they will:

• Gain a fundamental understanding of how this technology works and the impact it can have in AEC and the built environment by being exposed to example applications.

• Be able to identify limitations, pitfalls, and bottlenecks in these applications.

• Critically think on solutions for the above issues.

• Acquire hands-on experience in creatively thinking and designing an application given a base system, with in-class demos and at-home assignments.

• Use this course as a “stepping-stone” or entry-point to CV-ML intensive courses offered in CEE and CS.

Prerequisites / Notice

The course does not require any background in CV-ML, computer science, coding, or the AEC domain. It is designed for students of any background and knowledge on these topics. Despite being an introductory class, it will still engage advanced students in the aforementioned topics.

Performance Evaluation

The grading for this course will be a combination of assignments and a final project. Throughout the course students will be asked to work on assignments that would either require critical thinking, research in prior work, or hands-on interaction with a pre-existing system. The course also includes a final project. Students will be asked to creatively design and develop an application based on the material covered in the course lectures. Assignments are designed to complement the final project. The course does not have a final exam. Students can be evaluated with a letter grade or credit/no credit.

Schedule

Lectures

Deadlines

All the online submission deadlines are due by 11:59 PM on the specified date.

Student Projects

Over the quarter, students will work on a project related to a topic in Computer Vision for the Built Environment in collaboration with a supervisor. Students are required to form groups of 2-4. For this purpose, we will provide a list of project suggestions, but you are free to propose your own project. If you are familiar with coding and computer vision, you can implement an algorithm or test off-the-shelf approaches on AEC applications. If you are not familiar with either, you can conceptualize from beginning to end how you would go about creating such an algorithm (including but not limited to dataset collection, annotation, user testing, evaluation metrics, etc.). We will provide you with specific aspects that we want to see covered and discussed in your projects.

Project Proposal

Each student group is required to hand in a project proposal by the announced deadlines. Make sure to talk to your assigned supervisor and discuss the project with them while planning your proposal. The proposal should be 1-2 pages describing what you want to do in the project. A good place to start is to identify why this project is important to the AEC domain and what is the expected impact on different aspects (e.g., organization, safety, materials, user, data, etc.). Consider and include some preliminary thoughts on the criteria of success and the way you would evaluate them. Depending on class size, teams will be asked to present their project proposal during a designated lecture. In any case, you will submit a presentation file along with your proposal. We will provide both templates to follow. 

Midterm Progress Check

At this milestone, you are required to submit a 2-page report that identifies your current status, problems you have identified, and your next steps. Depending on class size, you will also present what you did so far so that you get feedback. You are encouraged to raise open questions. This is a possibility for us to steer the project and help you, as well as to get feedback from your fellow classmates. In any case, you will submit a presentation file along with your progress report. We will provide both templates to follow.

Final Project Delivery

You will present your final project at the last day of the course. You will also submit a final 8-pages report including tables and figures but excluding references, as well as the presentation file.  We will provide you templates.

Other Resources