Stanford Engineering
Gradient Spaces Lab

Computer Vision for the Built Environment

Course Information

  • Course ID: CEE 247C (Winter 2024)
  • Instructors: Prof. Iro Armeni
  • Teaching Assistants: Liyuan Zhu
  • Lectures: Wednesdays 1:30 - 4:20 PM in Y2E2 292A
  • Credits: 3 or credit/no credit
  • Online Material: Canvas

Course Description

Summary

The course is an introduction to Visual Machine Perception technology – and specifically Computer Vision and Machine Learning (CV-ML) – for the built environment. It will explore fundamentals in this technology both in research and products, in tight reference to design, construction, and operation/management. It will consider the current and potential impact of this technology on achieving sustainability goals, such as related to reuse, circularity, and performance-based lifecycle, as well as the organizational considerations behind development and adoption.

About

The past few years a lot of discussion has been sparked in AEC on CV-ML for the built environment. Despite advancements in this interdisciplinary field, we still have not answered fundamental questions about adopting and adapting CV-ML technology. In order to achieve this, we need to be equipped with rudimentary knowledge of how this technology works and what are essential points to consider when applying CV-ML to this specific domain.

In addition, the availability of sensors that collect visual data in commodity hardware (e.g., mobile phone and tablet), is creating an even bigger pressure in identifying ways that new technology can be leveraged to increase efficiency and decrease risk in this trillion-dollar industry. However, cautious and well-thought steps need to be taken in the right direction, in order for such technologies to thrive in an industry that showcases inertia in technological adoption and to drive sustainability goals.

The course will unfold in two interwoven storylines:

  • The first storyline will introduce fundamentals in computer vision and machine learning technology, as building blocks that one should consider when developing related applications. These blocks will be discussed with respect to latest developments (e.g., deep neural networks), pointing out their impact in the final solution.

  • The second storyline consists of AEC processes, namely architectural design, construction, and operation/management. These processes will serve as application examples of the technological storyline.

Students will see the importance of taking into account the application requirements when designing a CV-ML system, as well as their impact on the building blocks. Guest speakers from both the CV-ML and AEC domains will complement the lectures.

Learning Goals

By the end of the course students will develop computational thinking related to visual machine perception applications for the built environment and the architecture, engineering, and construction (AEC) domain. Specifically, they will:

  • Gain a fundamental understanding of how this technology works and the impact it can have in AEC and the built environment by being exposed to example applications.

  • Be able to identify limitations, pitfalls, and bottlenecks in these applications.

  • Critically think on solutions for the above issues.

  • Acquire hands-on experience in creatively thinking and designing an application given a base system, with in-class demos and at-home assignments.

  • Use this course as a “stepping-stone” or entry-point to CV-ML intensive courses offered in CEE and CS.

Prerequisites / Notice

The course does not require any background in CV-ML, computer science, coding, or the AEC domain. It is designed for students of any background and knowledge on these topics. Despite being an introductory class, it will still engage advanced students in the aforementioned topics.

Performance Evaluation

The grading for this course will be a combination of assignments and a final project. Throughout the course students will be asked to work on assignments that would either require critical thinking, research in prior work, or hands-on interaction with a pre-existing system. The course also includes a final project. Students will be asked to creatively design and develop an application based on the material covered in the course lectures. Assignments are designed to complement the final project. The course does not have a final exam. Students can be evaluated with a letter grade or credit/no credit.

Schedule

Lectures

DateLecture
January 101. Introduction
January 172. Drawing lines, surfaces, and primitives in visual data
January 243. As-is geometric model: From pixels to 3D reconstruction
January 314. Making sense of visual data: Segmentation and clustering
February 075. What is this that I see?: Visual data classification
February 146. Toward a "digital-twin": Detection and Semantic Segmentation
February 217. The machine designer: Generating new visual data
February 288a. Keeping track of mobile elements in construction sites: Object and people tracking
8b. Construction worker productivity and safety: Activity recognition
March 069. The machine worker: Human-Robot Interaction
March 1310. Final Project Presentation
  

Deadlines

DeadlineDescription
January 21Project proposal report and slides due
January 24Project proposal presentation in class
February 18Midterm project report and slides due
February 21Midterm project presentation in class
March 13Final project presentations in class
March 19Final project report and slides due

All the online submission deadlines are due by 11:59 PM on the specified date.

Student Projects

Over the quarter, students will work on a project related to a topic in Computer Vision for the Built Environment. Students are required to form groups of 2. We will provide a list of project suggestions, but you are free to propose your own project. If you are familiar with coding and computer vision, you can implement an algorithm or test off-the-shelf approaches on AEC applications. If you are not familiar with either, you can conceptualize from beginning to end how you would go about creating such an algorithm (including but not limited to dataset collection, annotation, user testing, evaluation metrics, etc.). We will provide you with specific aspects that we want to see covered and discussed in your projects.

Project Proposal

Each student group is required to hand in a project proposal by the announced deadlines. Make sure to discuss the project with us while planning your proposal. The proposal should be 1-2 pages describing what you want to do in the project. A good place to start is to identify why this project is important to the AEC domain and what is the expected impact on different aspects (e.g., organization, safety, materials, user, data, etc.). Consider and include some preliminary thoughts on the criteria of success and the way you would evaluate them. Teams will be asked to present their project proposal during a designated lecture. In any case, you will submit a presentation file along with your report. We will provide both templates. 

Midterm Progress Check

At this milestone, you are required to submit a 2-page report. Describe your progress on the topic w.r.t. the questions you should be answering. You don't need to show progress in all questions, but consider this as a good checkpoint about half-way before the final presentation and report are due. You are encouraged to raise open questions. You will also present in class what you did so far to get feedback. You are encouraged to raise open questions. This is a possibility for us to steer the project and help you, as well as to get feedback from your fellow classmates. In any case, you will submit a presentation file along with your progress report. We will provide both templates.

Final Project Delivery

You will present your final project at the last day of the course. You will also submit a final 8-pages report including tables and figures but excluding references, as well as the presentation file.  We will provide you with templates.

Other Resources

Book Suggestions: