Computer Vision for the Built Environment

Course Information

• Course ID: CEE 247C (Winter 2025)
• Instructors: Iro Armeni
• Teaching Assistants: Liyuan Zhu, Sayan Deb Sarkar
• Lectures: Wednesdays 1:30 - 4:20 PM in Y2E2 292A  (Spilker 232 only for Feb 5th)
• Credits: 3 or credit/no credit
• Website: https://gradientspaces.stanford.edu/computer-vision-built-environment
• Office Hours:
  • Iro Armeni, Tuesdays 2-3PM, Y2E2 233
  • Liyuan Zhu/Sayan Deb Sarkar, Fridays 11AM-12PM, Y2E2 278B
• Online Material: Canvas

Course Description

Summary

The course is an introduction to Visual Machine Perception technology – and specifically Computer Vision and Machine Learning (CV-ML) – for the built environment. It will explore fundamentals and latest trends in this technology both in research and products, in tight reference to design, construction, and operation/management. It will consider the current and potential impact of this technology on achieving sustainability goals, such as related to reuse, circularity, and performance-based lifecycle, as well as the organizational considerations behind development and adoption.

About

The past few years a lot of discussion has been sparked in AEC on CV-ML for the built environment. Despite advancements in this interdisciplinary field, we still have not answered fundamental questions about adopting and adapting CV-ML technology. In order to achieve this, we need to be equipped with rudimentary knowledge of how this technology works and what are essential points to consider when applying CV-ML to this specific domain.

In addition, the availability of sensors that collect visual data in commodity hardware (e.g., mobile phone and tablet), is creating an even bigger pressure in identifying ways that new technology can be leveraged to increase efficiency and decrease risk in this trillion-dollar industry. This is further accentuated by the off-the-shelf, generalization and zero-shot capabilities of large multimodal models that have become a daily commodity the past two years. However, cautious and well-thought steps need to be taken in the right direction, in order for such technologies to thrive in an industry that showcases inertia in technological adoption and to drive sustainability goals.

The course will unfold in two interwoven storylines:

• The first storyline will introduce fundamentals in computer vision and machine learning technology, as building blocks that one should consider when developing related applications. These blocks will be discussed with respect to latest developments (e.g., diffusion and large language neural models), pointing out their impact in the final solution.
• The second storyline consists of AEC processes, namely architectural design, construction, and operation/management. These processes will drive the application of the technological storyline.

Students will see the importance of taking into account the application requirements when designing a CV-ML system, as well as their impact on the building blocks. Guest speakers from both the CV-ML and AEC domains will complement the lectures.

Learning Goals

By the end of the course students will develop computational thinking related to visual machine perception applications for the built environment and the architecture, engineering, and construction (AEC) domain. Specifically, they will:

• Gain a fundamental understanding of how this technology works and the impact it can have in AEC and the built environment by being exposed to example applications.
• Be able to identify limitations, pitfalls, and bottlenecks in these applications.
• Critically think on solutions for the above issues.
• Acquire hands-on experience in creatively thinking and designing an application given a base system, with in-class demos and at-home assignments.
• Use this course as a “stepping-stone” or entry-point to CV-ML intensive courses offered in CEE and CS.

Prerequisites / Notice

The course does not require any background in CV-ML, computer science, coding, or the AEC domain. It is designed for students of any background and knowledge on these topics. Despite being an introductory class, it will still engage advanced students in the aforementioned topics. It is expected that students will engage with code and even implement their own code, however, this will be tightly supported by the tutorials and assignments.

Performance Evaluation

The grading for this course will be a combination of tutorials, assignments, and a final project. Throughout the course students will participate in tutorials that will engage them deeper in algorithmic and application aspects, in a hands-on manner. Evaluation on tutorials will be based on a set of questions that assesses the understanding of the presented material. Students will also be asked to work on assignments that would either require critical thinking, research in prior work, and/or hands-on interaction with a pre-existing system/codebase. The course also includes a final project. Students will be asked to creatively design and develop an application based on the material covered in the course lectures. Tutorials and assignments are designed to complement the final project. The course does not have a final exam. Students can be evaluated with a letter grade or credit/no credit.

The grading weights are as follows:  42% for assignments (6% per assignment), 6% for tutorials (1% per tutorial), and 52% for the final project (10% for the project proposal, 12% for the midterm, and 30% for the final submission). Evaluation criteria per tutorial, assignment, and project milestone are offered on Canvas under each relevant entry in "Assignments". When evaluating the course projects, we will assign projects for review to a member of our team that was not the assigned TA, to remove bias. We will also ask for quick feedback from your project supervisors and the assigned TA. The final project grade will take place by considering all feedback during a discussion with the teaching team. All members within a team will receive the same grade for their project, unless there are obvious signs or information of non-equal participation and work.

Schedule

Lectures

Assignment List

Deadlines

All deadlines are due by 11:59 PM on the specified date, unless indicated otherwise.

Student Projects

Over the quarter, students will work on a project related to a topic in Computer Vision for the Built Environment in collaboration with a supervisor. Students are required to form groups of 2-4. For this purpose, we will provide a list of project ideas you can choose from. We will expect you to implement/test the behavior of algorithms and discuss all aspects of, including but not limited to, dataset collection, annotation, user testing, evaluation metrics, etc. We will provide you with specific aspects that we want to see covered and discussed in your projects. Details can be found in Canvas.

Project Proposal

Each student group is required to hand in a project proposal by the announced deadline. Make sure to talk to your assigned supervisor and discuss the project with them while planning your proposal. The proposal should be 1-2 pages describing what you want to do in the project. A good place to start is to identify why this project is important to the AEC domain and what is the expected impact on different aspects (e.g., organization, safety, materials, user, data, etc.). Consider and include some preliminary thoughts on the criteria of success and the way you would evaluate them. Depending on class size, teams will be asked to present their project proposal during a designated lecture. In any case, you will submit a presentation file along with your proposal. We provide both templates and assessment criteria in Canvas.

Midterm Progress Check

At this milestone, you are required to submit a 2-page report that identifies your current status, problems you have identified, and your next steps. Depending on class size, you will also present what you did so far so that you get feedback. You are encouraged to raise open questions. This is a possibility for us to steer the project and help you, as well as to get feedback from your fellow classmates. In any case, you will submit a presentation file along with your progress report. We provide both templates and assessment criteria in Canvas.

Final Project Delivery

You will present your final project on the last day of the course. You will also submit a final 8-pages report including tables and figures but excluding references, as well as the presentation file. We provide all templates and assessment criteria in Canvas.

Other Resources

Book Suggestions:

• A Circular Built Environment in the Digital Age. (book) 
  Editors: De Wolf, C., Çetin, S., & Bocken, N. M., Springer
• Computer Vision: Algorithm and Applications (book)
  Richard Szeliski, Springer
• Computer Vision: Models, Learning and Inference (book)
  Simon J.D. Prince, Cambridge University Press
• Multiple View Geometry in Computer Vision (book)
  Richard Hartley and Andrew Zisserman, Cambridge University Press
• Deep Learning (book)
  Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press