UCSD Engineers For Exploration
EXPLORATION THROUGH TECHNOLOGY
Engineers for Exploration (E4E) is a one of a kind program centered around multidisciplinary and collaborative student research projects with the broad goals of protecting the environment, uncovering mysteries related to cultural heritage, and providing experiential learning experiences for undergraduate and graduate students. We team student engineers with scientists from a wide range of disciplines, such as ecology, oceanography and archaeology. Students create new technologies to aid these scientists in their work and then accompany them on field deployments around the world. Projects train students in embedded systems and software, machine learning, electronic integration, mechanical design, system building, as well as project management and team leadership.
Interested in joining?SPONSOR OUR RESEARCH
Your support allows us to involve more students, which allows us to continue our research on existing and future projects. It is easy and even a small amount can make a big impact. Donate To UCSD E4E.
Latest news
-
-
Winter 2026 Info Session
Info session will be held at CSE 1242 on January 15th at 1:00 PM!
Read more -
Listening Without Labels: Unsupervised Exploratory Data Analysis for Bioacoustics
Images are straightforward to interpret. They’re intuitive to human brains, and, owing to their prevalence, well-studied. Sound, by contrast, can only be experienced over time, requires undivided attention, and is often clouded by ambient complexity. While this complexity can make sound harder to analyze, it also carries ecological information that images miss. As such, bioacoustics have become an increasingly useful method to monitor ecosystem health in a cost-effective manner. AudioMoths, for example, are palm-sized recorders that can gather over 35 days of audio on a single charge.
Read more -
Introducing FishSense Scout
Factors like geographical location, water condition, and ocean depth can present significant challenges in diver-operated systems for fish ecosystem analysis. To overcome these hurdles, remote operated vehicles (ROVs) have presented themselves as valuable tools for collecting fish data in the lesser-documented areas of the ocean. Previous E4E systems, like FishSense Lite, target the monitoring of fisheries through citizen science. With the introduction of Scout, our horizons are broadened beyond waters safe for human deployment. Aside from aiming to increase accessibility to those under-documented waters, Scout also hopes to reduce the need for trained experts in the process and allow for longer deployments in broader areas, working towards the goal of more streamlined data collection. Scout explores the deployment of the existing fish detection, length estimation, and identification pipeline on embedded-edge computing devices paired with underwater camera technology for real-time computation of fish length and species. With a new underwater camera loaned to us from Deep Water Exploration that is specifically designed to correct for underwater aberrations, our first objective for Scout has been to collect and analyze data from this camera. We began by reviving an old watertight FishSense enclosure from a previous FishSense Lite product, along with the Jetson TX2 for the collection of camera calibration data. DWE’s StellarHD Aquagon, specifically designed for correcting for underwater optics, with a laser mount attached To calibrate and test the camera’s performance underwater, we have performed weekly pool tests for data collection. Students Theo Darci-Maher and Nykolas Rekasius took the camera along with the old FishSense enclosure into a pool and collected data for camera calibration. Students Annabelle Chen and Sijan Shrestha worked on the pool deck with their PhD mentor, Chris Crutchfield, to provide support and debug with the camera and the TX2. Theo handling the enclosure, pointing the camera towards the calibration board Example footage captured from the test, featuring Nyk holding the calibration board After this initial pool test, we analysed the collected camera footage and determined the need for a better calibration target. Hence, during our second round of testing, we used a segmented calibration board as opposed to a checkered one, making sure to position it in each corner so that we could later quantify the amount of distortion occurring at the edges of the frame. As a fair amount of color distortion was visible in the previous test (see above), we also incorporated calibration of a color card. Since motion blur was a possible concern in the results of the first test, we weighed the enclosure down and moved the calibration boards in front of it rather than moving the enclosure itself. The segmented calibration board, positioned to capture distortion in the lower left corner of the frame The color collection card positioned at the center of the screen After these first few pool tests, the TX2 stopped booting due to a flash error. In response, we swapped the TX2 out for Qualcomm’s Rubik Pi 3 to begin our work towards on-device computation. Because of the substantial power draw and, hence, generated heat from the board, our next pool tests focused on how to best manage the heat of the Pi within the underwater enclosure. The existing enclosure was adapted by Theo to accommodate for the size and structure of the new board, and its accompanying battery pack. To consider the maximum amount of heat that could be generated, we have run CPU stress tests on the board in different physical environments. During these tests, we monitored the onboard temperature and analyzed that data to consider the most effective and reasonably feasible option for heat transfer given the existing enclosure. The Rubik Pi 3 and its power bank attached to the enclosure's endcap The Rubik Pi enclosure setup, along with camera Plotted data from a 30 minute long stress tests Future work with Scout includes further stress testing of the Rubik Pi’s remaining processing units (GPU, NPU), analysis of our current camera data, testing in more realistic conditions, and potentially exploring other commercially available computer-operated cameras. With these changes leading to future deployments and collaborations, we will expand FishSense’s ability to improve accessibility of data collection in underdocumented ocean ecosystems! A (very official) post-pool testing debriefing session
Read more -
Bear-ly Audible Tracking Panda Vocalizations with STM32
Do you know what sound a panda makes? Over the last eight weeks, our team has been collaborating with the San Diego Zoo Wildlife Alliance to find ways to record, label, and store the various vocalizations produced by panda bears, polar bears, and several other species using specialized collars. While initially simple, the problem has posed several questions relevant to the field of computer hardware at large: how do you keep a low-power device running for periods of over a year? Can you collect only the relevant sounds, while avoiding environmental noise? What is the domain shift between a noisy MEMS microphone and high-quality training data? This project is split into several engineering problems and a larger, overarching design problem: collecting microphone data, denoising that data, running lightweight inference, and storing the result are all fairly trivial issues on their own, but they become more complicated when the resources are heavily constrained and the target audience–intended to both collect and access the data–is unfamiliar with the technology. In particular, power consumption optimization and ease of use are dissimilar–and, in some cases, incompatible–goals. Thus, the goal of this project is not only to develop a bioacoustic device, but to solve an optimization problem. This project was made possible thanks to the efforts of Milo Akerman, Francisco Irazaba, AnMei Dasbach-Prisk, and Hayden Dosseh, along with the teams who have worked on hardware and software previously. Crossing the STM Barrier The first issue our team faced was the system architecture. STM32 boards are, by design, more complex to work with than microcontrollers running on ESP32 or ESP8266. On one hand, peeling away layers of abstraction and letting you interact with the board at the register level (bare-metal programming) allows for lightning-fast operations and very low power overheads. On the other, it means you may be stuck for weeks debugging a linker file. The latter is a sacrifice more than acceptable for this project, but it did mean we would occasionally get stuck struggling against simple peripherals like USART or SPI. It was once we got a firm grasp of the STM32H7 that we realized the boon that was its low-level design principles. At the end of the day, we were able to optimize our peripheral usage without needing to swap boards or mess with solder bridges–and it was then that we could really get to work. Output spectrogram from on-board MEMS microphone. Notice the noise at mel bands 20-30, 60 20 Milliamps or Less If you want a board running continuously for a year on 6kg of batteries (just slightly less than 1% of the body weight of a female panda bear) you need, on average, a current draw of 20mA or less. Concerningly, previous studies placed the power estimates for a system like ours at about 100mA for inference alone on a TinyML model. Therefore, our first task was to build a system as power efficient as possible. This is how we arrived at our various prototypes for sleep systems, a task made easy by the STM32H7’s various inbuilt low-power modes and difficult by the lack of a good wake-up trigger. A convenient side effect of triggered wake-up is that, since the data is only collected when something of interest occurs, there is no need to sift through or store noise, thereby saving processing power, storage space, and much-needed time for the scientists. For weeks we combed papers and scribbled over our whiteboard, trying to determine what the best way to issue a wake-up signal was, before arriving at two possible solutions: we either implemented digital signal processing (DSP) on the board’s second, less power-hungry core (the Cortex-M4) or we designed and stress-tested an analog signal processing circuit. Left: System diagram of our collar, with simplified piezo wake-up circuit (analog signal processing) at bottom. Right: A similar, complete piezo wake-up circuit. Marzetti et al., 2020 Due to the complexity of the latter system, along with a lack of in-house prototyping material and a wealth of possible failure modes, we decided to go for the DSP approach. We settled on a system which performs very basic operations on the M4 core, storing the audio data in shared memory with its sister M7 core. This allows us to switch between the two quickly, only ever having one active at a time, while relaying the data seamlessly. We conducted two power studies on our system. One early on, which showed high figures mostly due to unoptimized code and improper clock frequencies, a second one with full peripheral integration. By the end, our system was showing promise in terms of power consumption, allowing us to deploy for durations exceeding both the original estimates and the project specifications. First Study Second Study Baseline - Wake 13.43 mA 6.99 mA Baseline - Sleep 2.9 mA 2.7 mA Mic + SAI + DMA 23.6 mA 10.37 mA DSP + Inference 16.0 mA 12.46 mA MicroSD Write 14.1 mA 7.74 mA Better-than-Algorithmic Compression Finally, we arrive at another important issue: how do you store data securely, for a year or more, under restrictive and unpredictable conditions? There is a simple solution: SD cards are small, light, and mostly lossless, in addition to having fast and power-efficient write cycles. The biggest issue with this approach is that data can’t be collected until the deployment is complete, a feature that can prove very inconvenient, depending on the use case. Given that our collaborators have an already established LoRa (Long Range) network, we did a lot of work trying to make our system integrate wireless communications as an alternative to SD storage. Famously, wireless communications are very power-hungry. LoRa uses less power by transmitting data through “chirps”, at the cost of significantly lower bandwidth and longer uplink times. More importantly, LoRa is still by far the most power-intensive of our system. To reduce this issue, and to avoid blocking the network for too long, it quickly became apparent that we would need to compress our audio files significantly. In fact, if we reduced it enough for a full transmission, we could avoid running inference on-device at all, and classify the sound files with more powerful compute resources elsewhere in the network. For now, we decided to see how far we could take the compression. And, after implementing SD writes with FATFS, we stumbled upon an interesting–and promising–alternative to audio codecs: autoencoders. The idea behind autoencoders is as follows: a multi-layer perceptron (the encoder) is trained to compress, for example, an image of a spectrogram into a set of embeddings, which another MLP (the decoder) can then decompress into a reasonably accurate representation of the original image. If you sever the two parts of the model, you get a section which can encode images into a smaller representation (known as the “latent space”) and a section which can decode it to its original state. Why would you use this, instead of compressing your audio algorithmically (e.g., MP3)? In theory, an autoencoder could reduce the amount of information significantly more than an algorithm by ignoring aspects of the audio irrelevant to classification and only focusing on the features that distinguish two spectrograms you care about. Sample autoencoder input and output, spectrograms courtesy of Anu Jajodia Putting everything together, we arrive at a cutting-edge device capable of acting as a low-power edge node within and without an external network: a versatile device for analyzing animal behavioral data, which ultimately has the potential to aid countless conservation efforts. We hope our work will be of use to the San Diego Zoo, and continue to be iterated on for years to come.
Read more -
Come See the Summer 2025 REU Final Presentations
Engineers for Exploration is once again hosting final presentations for the REU students this year! This year will include recent work and effort done by the REU students working in exploratory methods for audio data mining, development on edge devices for acoustics, improvements to the FishSense System, improvements to secure chip access, and updates on the latest Smartfin device.
Read more -
2025 Summer Research Students
This summer, we are hosting 14 students from around the world in our 2025 summer research program at UC San Diego. Meet our students by reading their bios below:
Read more -
Congrats to Sean Perry for becoming an SDZWA Fellow!
PhD student and Acoustic Species Project lead Sean Perry was awarded the Conservation Technology Fellowship from the San Diego Zoo Wildlife Alliance!
Read more -
Kendall-Frost Marsh Acoustic Deployment!
South of UCSD lies Mission Bay, a popular tourist spot once a biodiverse-rich marshland destroyed through a combination of dredging and the redirection of the San Diego River. However, recent efforts have begun to preserve and restore some of the original habitats, particularly in the Kendall-Frost Marsh, one of several biodiversity reserves managed by the UC Natural Reserve System. This 40-acre marshland lies in the northern section of Mission Bay and is a popular spot for migrating birds and two endangered species: “Ridgway’s rail and Belding’s savannah sparrow” [1,2].
Read more -
Winter 2025 Info Session
Info session will be held at CSE 1242 on January 14th at 2:00 PM!
Read more -
Acoustic Species ID goes to NeurIPS 2024!
Our work, “A Deep Learning Approach to the Automated Segmentation of Bird Vocalizations from Weakly Labeled Crowd-sourced Audio” was accepted and presented at NeurIPS 2024 in the “Tackling Climate Change with Machine Learning” Workshop hosted by Climate Change AI! Congrats to the authors: Jacob Ayers, Sean Perry, Samantha Prestrelski, Tianqi Zhang, Ludwig von Schoenfeldt, Mugen Blue, Gabriel Steinberg, Mathias Tobler, Ian Ingram, Curt Schurgers and Ryan Kastner.
Read more -
Engineers for Exploration (E4E) Summer Research Program at UC San Diego - Application Submission by February 15th, 2025
UC San Diego’s Engineers for Exploration Summer Research Program is an NSF REU (Research Experiences for Undergraduates) Site centered around multidisciplinary and collaborative student research with the broad goals of protecting the environment, uncovering mysteries related to cultural heritage, and providing experiential learning experiences. Our program is a full-time paid research experience from June 23rd to August 29th in which students will work in multidisciplinary research teams to aid scientists from the San Diego Zoo, Scripps Institute of Oceanography, and UC San Diego in tackling problems in fields such as ecology, physical oceanography, and archeology. During this program, students will create and apply technologies in novel ways to aid scientists in their work, and may have opportunities to accompany these technologies on field deployments around the world. Through these projects, students can expect to learn about embedded systems and software, machine learning, electronics integration, mechanical design, system building, as well as project management and team leadership. For more information, please visit our website at https://e4e.ucsd.edu. Applications are currently being accepted at https://e4e.ucsd.edu/apply through February 15. Applications will continue to be accepted through March 30 if positions are not yet filled. For questions about the program, email e4e@ucsd.edu.
Read more
subscribe via RSS
