Monday, April 1, 2013

Farewell for now from Cal Poly!

It feels like this month has gone by so quickly! But either way, the time came for us to bid adieu to the Maltese Islands. From interesting cisterns, to collaboration with Harvey Mudd; from interesting cultural tours, to once-in-a-lifetime opportunities; this experience was unforgettable for each and every one of us. Just to name a few of the highlights were Timmy's never-ending knowledge of the Maltese Islands, Davide's tours of the Siracusa area, and the wealth of intriguing talks at our apartments, on topics including the Maltese language, Maltese politics, robotics control systems, subterranean Valletta, ancient water systems, and cultural heritage management.
Although our Mediterranean excursion has come to an end, we certainly have more work ahead of us, with the conclusion of our individual research projects, and quite a bit of data processing as a team. Thank you to all of our collaborators, supporters, suppliers, and followers, and stay tuned for progress and wrap-ups on our projects!
Aerial view of Gozo from our Air Malta flight
Stateside welcome upon entering customs at LAX
Late night view of the San Luis Obispo airport upon returning



Thursday, March 28, 2013

Malta vs. Italy FIFA World Cup Qualifying Match

The ICEX team outside the stadium

While we were back in the U.S. the whole team did some research on cultural things we could do while we were in Malta. During our researching, Andrew discovered that Malta and Italy were going to play a FIFA World Cup Qualifying Match during our stay and we could not pass this great opportunity. On Tuesday night, after mapping the St George Bay cave, the time finally arrived
and the whole team was super excited for the match. Even though Malta did not win, the cheering crowd was not affected by the overall score. We even got to do the Mexican wave!


Ian and Amanda cheering for Malta

Fans forming the Maltese flag

Jeff, Andrew, and Erik entering the stadium





St. George's Cave



On Monday and Tuesday we visited St. George’s Cave. When we arrived at the cave, we discovered that the water level had risen significantly since the last deployment 2 years ago. The land bridge that had previously connected a large rock hill was approximately 1 foot underwater, so we were forced to setup our control box and other equipment outside of the cave.

Rock hill on the right
More of the cave...Pictures just don't do it justice in scale
We quickly deployed with the Smart Tether and began taking sonar scans.  It took a couple of hours just to map the outer walls of the cave, and it was lunchtime before we knew it.

After lunch, we redeployed with the HOBO sensor. When we went to put the ROV in the water, we noticed that the water level had risen since the morning. This made us wonder if the cave is somehow connected to the sea. Our HOBO data may help us answer this question.
Andrew on tether

Tape for our "Slices of Pie"
We quickly ran into a number of Smart Tether issues. After about an hour of troubleshooting we decided to begin taking manual measurement by driving the VideoRay to a wall, marking the tether with tape, diving to the bottom, and marking again. We kept a log of time stamps and approximate locations in my journal, in addition to our usual log books. These timestamps will help us match up the corresponding HOBO data. We decided to call this our “Slices of Pie” method, since we were taking new readings ever few degrees around the semicircle cave.



By  By the end of the day, we had finished collecting dense HOBO data for about half of the cave. When we returned on Tuesday, we decided to continue our “Slices of Pie” method for the remaining half of the cave. Interestignly, about 1/3 of the cave was blocked from view by the rock hill mentioned above.


View from the cave entrance
I quickly volunteered to be the adventurer who would cross the submerged land bridge to assist with the measurements on this portion of the site. As soon as I had crossed the bridge, we began having some software issues, so I spent ~20 minutes alone in the dark cave. It was a little creepy at first, then quite peaceful, and then very fun, as my eyes slowly adjusted and I explored more. By the end, I felt like Gollum sitting inside a dark cave surrounded by water.

Once the ROV software was back up and running, we quickly finished our slices of pie. One of the slices had a deep channel below it. We dove about 20 meters and could not descend lower, but still saw pitch black below, indicating that it went much further!

I’m excited to see the results of this deployment. Even with our manual logging style I am hopeful that we will get some interesting results.

Wednesday, March 27, 2013

Mapping and Visualization Updates

Hi everyone! We've been working hard on producing maps and visualizations of some of the cisterns. I have been particularly interested in reconstructing medium sized features from stereo camera data (archways, stairs, bumps, etc.) in the Mediterranean Conference Center cistern that I made a sonar mosaic of the other day. To affix our stereo data to the rough SONAR generated models, first we have to make a map and reconstruction of the cistern.

The cistern was enormous, and our software can only support chunks of the larger models at any one time (well... before we use up 8 gigs of RAM and crash our software), so I have been working on the cistern piece by piece.

I captured some great stereo images in one of the main chambers of the cistern. From the 2D sonar mosaic the room doesn't look too interesting. However, from a 3D standpoint the chamber looks like a mushroom! Much more interesting! This is why 3D SONAR is so important to us and to our archaeologist friends.


Mediterranean Conference Center 2D Sonar Mosaic



Mediterranean Conference Center 3D Sonar Mosaic


I took the 3D sonar mosaic and converted from pixel space to 'blender space', and then from 'blender space'  to meters. I then loaded the scans into our mapping algorithm to produce a 3D occupancy grid of the SONAR data. Here is the map which we will use to construct the final 3D mesh which will be a closed surface and will look much nicer!


Mediterranean Conference Center SONAR Occupancy Grid

So far this room in the map is about halfway through our visualization pipeline. We have a couple more steps to go in order to make it a pretty 3D graphic that people can use for study and learning. I intend to fuse some of the stereo data from this cistern into the model as well. The work on that front has been a little more difficult due to the underwater image noise and poor quality of our stereo hardware, but here are a couple of preliminary disparity maps that were processed (in the method described a day or two ago) from raw underwater captures on the stereo cams.



The plan is to add some of these features back into the meshes, so stay tuned for results on the computer graphics end in the next couple of days!


Footnote: Just for fun - Vanessa just posted this, but here is a picture of me driving the robot in this very cistern (in the deep dark depths of the paint closet) to give a little bit of context.


Saturday Deployment with HMC

Last week was an exciting week with the Harvey Mudd students and faculty! We were sad they couldn't stay for longer, but were happy to have their help and expertise for the time they were here.

Working with the HMC Students
On our last day together, we went to MCC for the third time to use double sonar (vertical and horizontal scans). We were successful in using this new technique to map the cisterns in 3D. The cisterns at MCC were really complex and we are excited to have some models made by Erik and Ben! Given that we are from different campuses we had only met once before the trip, but we quickly learned how to work together efficiently and effectively! We hope to visit HMC next quarter to share our results and experiences.

Deploying the ROV (Erik, Spencer)
And, of course, our wonderful professors grading!
Zoe and Chris

Tuesday, March 26, 2013

Final thoughts

The Harvey Mudd ICEX team (Chris, Alistair, Joshua, Ben and I) wrapped up our trip and are now stateside. Minus a couple pieces of luggage, we made it back in the middle of the night and in time for Chris' autonomous robot navigation class.

The Maltese coastline alongside Valletta
All in all, this trip has a great experience - this includes learning about the rich history of Syracuse and Malta, having the opportunity to access private properties and catacombs to conduct research, and the chance to play the many roles needed in each deployment. To run a deployment and document our work, the team needs someone on the log books, the ROV tether, driving the ROV and collecting sonar data. Here are some great shots Zoe got from our last HMC-Cal Poly deployment in Valletta -

Vanessa and I on tether.
Driving the ROV and collecting data
during a deployment.
It wasn't hailing this time, and on these last few deployments we got some great data from 3D sonar data of some really intricate cisterns and deploying both ROVs!

Our 3D sonar set up on the ROV :)
Now that the cistern data collection has ended for us, our next steps are to map and localize the ROV using this information, and we also hope to work with younger students in Southern California to inspire the next generation of computer scientists and engineers with our work and outreach efforts.
The complete ICEX team, and our gracious dinner host.
I feel very lucky to have had the opportunity to be a part of this project. I want to thank the Cal Poly students for their work all month and getting us on board during our time in Malta/Sicily, our advisors (Jane, Zoe and Chris) for their guidance and great company, Timmy for enriching our trip by putting our work in a cultural and international context, and my teammates for pretty much everything. I wish the Cal Poly team luck in the rest of their work this coming week in Malta :)

Monday, March 25, 2013

Tentative Conclusions


I set out at the beginning of this journey with the goal of researching some of the most important natural resources to Malta. So far, I have investigated water, oil, coal, and stones. I would still like to research a number of additional resources such as wood and metals. I am also very interested in discovering what Malta does with its waste and what products it recycles. I have done a significant amount of research on this topic already but have not yet had time to write a nice summary.

A number of different research questions have become visible during my research. I highlighted some of them at the bottom of each post. For example, I am interested in researching how much water is imported into Malta in the form of food products.

I am also interested in exploring the sustainability of the current natural resource usage. I found some interesting information about limestone depletion and would like to continue researching this topic.

Even though my research is far from complete, our Maltese trip is quickly coming to a close. I am attempting to visit the University of Malta during my last week so that I can speak to some students and professors about my research topics. Specifically, I am very interested in their opinions on how a lack or abundance of certain resources has affected the development of engineering projects in Malta.

Outreach Research Project Update

As mentioned previously in one of Cecilia's update posts, we (Ceci and Ian) are researching the topic of evaluating one-off outreach education events. There is a surprisingly large body of research related to this question. In addition to this research we are trying to evaluate our outreach events both here (Malta and Sicily) and back in the states, applying the techniques and ideas from our research.

Ian during an outreach event

As a major part of our attempt to evaluate our own outreach events, we are developing a website to provide followup lessons for the students we teach. This website will contain videos about computer science, links to other resources, and lessons in the form of a simulator which Ceci is designing and developing for a graphics class project.

Now we're moving forward with integrating this simulation into a website and adding the videos and links. In order to develop the website we've set up a temporary web server to try things out online and a locally installed web server for quicker development and so we can still work when we don't have internet access.

In addition to the website for student interaction we're conducting a survey of the teachers for their evaluation of how the students learned in the outreach events. This survey has been completed, set up, and tested so it will be sent out to the teachers within the next week. Our Malta outreach has been completed and we have one more Sicily outreach event tomorrow.

In general, our evaluation process will involve self evaluations which we will compare against the teacher responses to our surveys. We'll also be asking the other ICEX members involved in the outreach events for their reflections both the specific events in question and of ways to move forward with robotics outreach events in general.

With regard to our formal research of the general evaluation topic, as mentioned before there is a large body of research in this feel. We have collected a number of large papers on the subject and are currently working our way through all of them. We'll be writing a report on the different techniques for evaluation along with our specific evaluation of our outreach events, and plan to have a first draft done by the end of the trip.

--

Ian and Ceci

Checking out the University

Today was a great day! Amanda, Andrew, Cecilia, Ian, Spencer, and Vanessa took a trip to the University of Malta to do some touring, research, and interviewing. (And a little shopping, of course!)


Upon arriving at the University, we headed straight to the Archaeology Center to meet up with Dr. Timmy Gambin, who is a Senior Lecturer in the Department. The Archaeology Center was a cool old stone building--something we are definitely not used to seeing at Cal Poly.


Timmy showed us all around the department, including the archive room that is full of archaeological findings and results from years of excavations.


After that, we got to see the student center and some of the buildings. We went to the Engineering Building, and got to check out the different disciplines of Engineering offered at the University here, including Systems & Control Engineering, which we learned about earlier this month from a Professor who got his Ph.D. working in robotics.

We loved all of the views and the courtyards at the campus. Everything was so green!


Then, Andrew went to the library to read up on politics throughout Malta's history and I interviewed some Professors about the Maltese attitude toward their water shortage and plans for addressing it.

On the way back, we picked up some shirts for the Malta football game tomorrow night! We're all really excited. Go Malta!

Sunday, March 24, 2013

Archipelago of Stone

The Maltese Islands have few natural resources; so many raw materials must be imported, with the exception of limestone. The islands are composed of sedimentary rock, which means that they have been formed over millions of years through the process of sediment compaction (Vella).

Examples of Globigerina and Coralline Limestone
Malta’s rock layers can be divided into five distinct types:
  • Upper Coralline Limestone
  • Greensand
  • Blue Clay
  • Globigerina Limestone­­
  • Lower Coralline Limestone


The oldest of these rock types (Lower Coralline Limestone) was formed roughly 35 million years ago, while the newest (Upper Coralline Limestone) was formed only 7 million years ago (Vella). In comparison, the majority of the exposed rocks in the Grand Canyon range from 200 million and 2 billion years in age (Grand).

Hagar Qim Temples
Both types of Coralline Limestone were formed from the skeletal remains of algae, corals, and molluscs. Globigerina Limestone was formed by the compaction of the remains of a single-celled organism called globigerina. It is believed that the ­Blue Clay layer may have been created from detritus deposits from Italian rivers. The Greensand layer was likely formed in a similar manner to the Blue Clay, but at different depths and with different water current conditions (Vella).



Model of the Mnajdra Temple Site
Tarxien Temples
The inhabitants of Malta have been using these abundant resources for thousands of years. We have been fortunate enough to visit several ancient temple sites, which are constructed entirely of stone, including Hagar Qim, Mnajdra, Tarxien. These sites date from 4,000 – 2,500 BCE. This makes them older than Stonehenge and the Egyptian pyramids (About, Timeline). I found the temples to be impressive in both their architecture and scale.

Limestone has continually been used in the construction of many types of buildings and sculptures. All of the baroque churches that we have visited have also been constructed of limestone. All of the fortresses built by the Knights of St. John were constructed with stone. Additionally, most modern buildings in Malta still utilize limestone extracted from Maltese quarries.

View from Fort Manoel
Arches at the gate of Fort Manoel









Coralline limestone is much harder and more durable than globigerina limestone, but this also make it more difficult and expensive to quarry. For this reason, the majority of modern buildings are constructed using globigerina, with coralline being applied in specific areas for additional strength (Cauchi).



Malta has a long history of exporting limestone to other countries for use in building materials and sculptures. Throughout the nineteenth century, limestone was traded with a number of Mediterranean countries including Greece, Turkey, and North Africa (Retention). Limestone has also continued to be an important export from Malta in the modern era (2011). It is exported to several nations in the European Union and elsewhere.


I am interested in researching the other building materials that are currently being used, such as steel and glass. I am interested in discovering whether these materials are now being used due to reduced cost, increased strength, environmental concerns, or a combination of these factors. Finally, I am interested in visiting/researching a quarry in Malta. The team visited two quarries in Sicily, and I found them and their history to be fascinating. I don’t think that we will have time to visit one during our trip, but it will definitely be at the top of my list if/when I return to Malta.

“About Stonehenge.” http://www.stonehenge.co.uk/about.php.

Cauchi, Rudolph.“The Maltese Limestone.” Limestone Sculptures. http://www.limestonesculptures.com/limestone.html.

“Grand Canyon Geology.” Grand Canyon National Park Lodges. http://www.grandcanyonlodges.com/grand-canyon-geology-5451.html.

“Retention of the Status Quo Regarding the Exportation of Maltese Stone.” Ministry for
Economic Services, Malta. 2000. http://www.lino-bianco.com/otherprojects/images/3_exportation/retentionofthestatusquo.pdf.

“Timeline of Ancient Egypt.” National Geographic. http://www.nationalgeographic.com/pyramids/timeline.html.

Vella, Leslie. “Living Stones: a brief guide to Malta’s geology.” Malta Inside Out. http://www.maltainsideout.com/11993/living-stones-a-brief-guide-to-maltas-geology/.

"The 2011 Import And Export Market For Worked Building And Monumental Stone In Malta." Country Trade Reports (2011): N.PAG. Business Source Premier. Web. 24 Mar. 2013.

Stereo Progress

Recently I've been spending time processing the data from our GoPro stereo cameras. I've have written most of the algorithm/backend processing that we need to reconstruct geometry found in the cisterns from stereo images, so most of the remaining work is actually processing the raw images captured in the cisterns to form something called a disparity map. I'll explain more about disparity maps later.

Our GoPro cameras:


I began by working on a set of pictures that were taken above the water surface but were still part of one of a cistern. I have processed a ton of photos now, and this is just one example.

Unfortunately, our GoPro cameras have a bit of a fish-eye lens, which can lead to some complications below the water surface. Last year, Tim and I corrected for this fish eye by dropping a checkerboard pattern into the pool, taking pictures, and flattening the image until the checkerboard was aligned with the dominant axes on the screen. A few other steps were applied to each image before rectification. To the left is an original image from the right GoPro camera.




After doing some basic preprocessing, we attempt to align the stereo images along epipoles in order to aid in stereo matching. To do this, I am using a MATLAB implementation which is freely available.



Here is a composite Red/Cyan image of the left and right images, respectively.









To rectify the images, we search for 'significant features' which can be matched between left and right. Then, we apply a transform on each image in order to align all of the significant features on top of one another. To the right are some feature identification pics. Note that not all features have a match between images. To match features, we do some more processing.



Some SURF features and their weights shown in green circles.
 Once features are identified and refined, we match significant features between left and right images. Notice how the features in each image are slightly offset from each other.




 Once we have features matched, we can apply a 2D affine transform to align the images along their stereo epipoles. This makes the disparity mapping algorithm easier later on (now we only have to search for similar features to our left and right, rather than up and down!).





Once the images are rectified, you can save the red and cyan channels as the new left and right images as input for the disparity matching algorithm. When we run the disparity algorithm, we get something that looks like this. Red things are close, blue things are far!



When you get a disparity map like this, you can do some pretty cool things. For example, you can now make x,y,z points in 3D space using the x and y values in the pixel coordinates in the original image, with the z value obtained from the color in the disparity map! Here is a cool example of what I mean.



Today I was able to get all of this processing scripted, so now I can click 2 buttons and process all of the images from a deployment into disparity maps. These disparity maps will be used for some exciting things, including reintroducing medium and fine details back into the cistern which were originally omitted from the low resolution SONAR data! This is just one half of my project, and I have a good deal of the second half complete already. Now it's time to combine the two and get some real-world results!