Activity 6 - Post Processing

GEMs Sensor - Sentek System Product Review / Report

Basic Overview 
     For the longest time imaging using a UAV meant making sacrifices based on what sensor you had available. Often this meant mounting a camera in an unorthodox position and asking it to do things it wasn't made to do. This all changed with the GEMs sensor by Sentek System. This sensor is one of the first of it's kind, a sensor made specifically to be used with a UAV. GEMs stands for Geo-localization and Mosaicing System and it happens too do both quite effectively as you can see here. The GEMs manages to pack a dual 1.3 megapixel sensor into a very small form and includes GPS too geocode each image taken, an almost absolute must when it comes to effectively image an area. The Ground Sampling Distance or GSD is 5.1cm @ 400ft and 2.5cm at @ 200ft. These figures are important because with aerial imaging, location accuracy is key. Something that sets the GEMs apart from most sensors is the way images are stored. Most cameras use SD cards but the GEMs saves images to an external usb flash drive. Using a flash drive is nice because the cost associated with these is lower, they can write faster than most other storage formats and hold more data. Because the GEMs is lighter than most sensors (170 grams) and smaller it makes it possible to mount in even the smallest of spots but there are a couple of things to look out for and consider when placing the sensor especially on quadcopters. The biggest thing to remember is that the GEMs is meant to be placed downwards to the ground and flat to minimize distortion. Some other factors to consider is the amount of vibration the sensor experiences. If the sensor vibrates too much the accelerometers in the sensor can be thrown off, reducing the quality of your finished mosaic. Something else to watch out for is EMI or electromagnetic interference, luckily Sentek came out with a solution and includes shields on the sensor to prevent that from happening. When it comes to actually flying your mission and recording data there are some factors to consider. Each sensor is different and you have to find the perfect mix of height, speed of flight combined with your sensor specs to get the level of detail your project requires. When using mission planner software there are some things you need to input regarding sensor information. Fortunately, Sentek provides information pertinent to the GEMs for Mission Planning. (Figure A)

Figure A - Values to input in to Mission Planner for best results.

Software Overview

     There also happens to be a GEMs software package available for purchase or comes alongside with the GEMs sensor. The software takes your taken images from the usb drive along with GPS coordinates and according to Sentek Systems creates an orthomosaic of multiple forms including RGB, NDVI, NIR. In this case the GEMs software creates a georeferenced image which is great but is really only usable for measurements when GCPs (Field Activity 4) are used. Georeferenced Images

Using the Software

     Using the software is pretty easy. Locate the folder where your images are located and insert them in to the GEMs software. The software automatically names the folder in this format - " Fight Data (Week=X TOW=H-M-S). Where X, H, M, and S are numbers specifying the instant that data collection began for the associated flight. X is the GPS week number of the starting instant. H, M, and S represent the hours, minutes, and seconds, respectively, into the GPS week of the starting instant." Using converters this can be turned in to an exact time frame. Once locating the folder click "run" and click "NDVI Initialization" Once this is finished running it can be imported into various programs such as ArcMap which is what we used to look at and work with the data. Once processed, bringing .tif images in to Arc allows you to match up the data you collected with a basemap and essentially covers the old imagery allowing you to see an up to date image of the area you imaged. This format is needed because .jpeg images do not include geolocations, essential for GPS accuracy. One thing to note with the data brought in using the GEMs is that the data does not have metadata associated with is so the user, in this case (me) has to go in and edit the metadata to preserve data integrity and provide information so others can see things such as when the data was collected and how it was collected.

Types of Images

     The GEMs is unique in the sense that it doesn't limit you to just RGB and modded NIR. There are several formats included with the GEMs, RGB, NIR and NDVI.

RGB - Figure 1 - Represents true color, what the surface looks like with your naked eye.

Figure 1 - RGB Image

Monofine Image - Figure 2 - The fine NDVI mono shows the reflectance levels as a high to low. The GEMS software does not assign a color scheme to this NDVI. Healthy high reflectance areas are white and low health is gray to black.

Figure 2 - Mono Fine Image

NDVI FC 1 - Figure 3 -  Mainly used for monitoring vegetation, the red and orange representing vegetation and water content of that vegetation.

Figure 3 - NDVI - FC 1 Image

NDVI FC 2 - Figure 4 - Much like FC 1 except more visually appealing, where green represents vegetation much like the color of the vegetation. Red is min. NDVI while Green is max.

Figure 4 - NDVI - FC2 Image

Normal NDVI Mono - Figure 5 - Black represents min. NDVI while White is max. this colorway is mostly used for finding small plant growth found in soil throughout a large area.

Figure 5 - NDVI Mono Image

Flight Path - Figure 6 - This map shows the mission and path the Matrix Quadcopter took carrying the GEMs in order to image this particular area at the Eau Claire Soccer Complex.

Figure 6 - Flight Path

Final Critiques 

     Anytime you purchase a first generation device you have to be willing to accept there may be flaws and often a high upfront cost, especially when there are not many other options available specifically for this application. The GEMs sensor is no different. Alternatives to the GEMs Sensor are cheaper, but are not necesarily created for UAVs yet people have found ways to use them, avoiding the high price tag of the GEMs. To purchase the GEMs you have to contact Sentek Systems directly and they will assist you with purchasing. As stated on their website the GEMs is 100% designed, tested and manufactured in the United States.
     The price set aside the GEMs is a very capable sensor and does what Sentek claims it does. Although only 1.3 megapixel the images are clear and effective but for agricultural purposes this is good enough. I'm a little disappointed they did not include a better camera just knowing with how advanced phone cameras have gotten considering the size factor.  Sentek markets the sensor for agriculture and as you can see in the processed images here, it will do an effective job. As competitors enter the market and availability increases the cost of sensors such as the GEMs will decline.
     If you have the funds and do not want to fiddle with modifying other sensors to accomplish what the GEMs does it will serve you perfectly. The GEMs plugs directly in to the UAVs power and works fluidly. We did not have any issues with processing the data and it was very capable. I would recommend the GEMs Sensor as of right now and will revisit this review if I feel otherwise once processing data with the other sensors. It is a sensor made for UAVs and delivers what Sentek claims, the software it comes with pairs with the sensor and prepares the captured images in to .tifs in a timely matter.  

Activity 5 - Obliques for 3D - Model Construction

Introduction:
So far this semester we have been capturing our images in Nadir or straight down below the aerial device. This works well because the scale of the image remains relatively constant so measuring distances is possible. This week we focused on taking oblique images to process and create a 3D - model. Shooting oblique images means having a tilt of more than 3% and angled compared to nadir where the image is straight vertical down. The oblique images can be stitched together to form an accurate, explorable 3-D model. For this field outing we focused on collecting oblique images to create just that, a precise 3-D model.

Study Area:
This week our study area was the Eau Claire Sports Complex. We specifically focused on the imaging the pavilion found there (Figure 1). This was a familiar location to us but this time we focused on collecting a set of much different images for processing later this semester.

Figure 1 - Pavillion we focused on imaging

The weather during our outing was excellent with it being around 60 degrees, light cloud cover and low winds. There was mare tail clouds in the sky indicative of a storm coming but luckily we avoided any storms that were on their way.

Methods:
The first set of data we collected was done so by the use of the 3DR Iris + accompanied with a GoPro camera. The GoPro is great for capturing video but not ideal for this purpose because there are no geotags associated with each picture taken. This makes it more difficult to make a model out of the images because there is no GPS associated with the images. We will have to match up GPS points from the flight log with the images in order to create a decently accurate model. Using Mission Planner we created a mission plan that is referred to as a "structure scan" this meant the Iris + would autonomously fly itself around the building starting at 5m and gather images every 2 seconds capturing as many angles and views of the structure as possible. Upon doing so we decided that we need to gather images lower than 5m so we did so manually after the autonomous mission was complete. Although our model is not complete yet I will post it when done. Here is a .gif image (Figure 2) of our flight so you can see the number of passes and corkscrews the Iris + did to capture all the angles of the pavilion.

Figure 2 - All the images gathered using the Iris +

The next set of data we collected was using the DJI Phantom 3 Professional. The Phantom was a little more suited for doing this task because unlike the Iris, the Phantom does geotag each image you take making it much easier to create a model with after data processing. With the Phantom all of our images were gathered manually. We each got a chance to fly around the pavilion and gathered pictures every couple of seconds. The .gif image of our flight can be seen below (Figure 3)

Figure 3 - All the images gathered using the Phantom 3

It will be interesting to see the difference between the two models. The Iris was flown almost strictly autonomous with very precise movements and regularly spaced image capturing (every 2 seconds) while the Phantom was manually flown and the images were all taken by the pilot at an irregular time interval. While doing previous field activities we focused on capturing a broad area for the purpose of mapping while with this activity there was emphasis on one area and making sure we were extra thorough in gathering images of one structure.

Discussion / Results:
Before today the only images we had captured of the pavilion looked like this (Figure 4) as you can see our images only focused on the roof. Now using images such as this (Figure 5) we can put together a 3 - D model that focuses on all surfaces and sides of the pavilion. With capturing the roof it really only comes down to needing one image if capturing in nadir but with oblique it takes many passes and varying angled shots to capture the whole roof. Capturing oblique takes a lot more time to cover an entire surface but the level of detail is significantly higher. In previous field outings we've had a strong focus on mapping but with this exercise we weren't necessarily mapping but more so modeling.

Figure 4 - Vertical image taken from Field Activity #3

Figure 5 - One of many oblique images taken of all sides of the pavilion

Conclusion:
Using two different devices we captured the same structure. Using two different methods not only increases our chances at getting a successful model but also shows us there are varying ways to accomplish the same goal. Using GPS data or geotagged images we will be able to create a 3 - D model of a structure using image processing software. The applications for this are really endless, farmers could use the same methods to image their fields, insurance companies for disaster areas, the fire and police services could model crashes and fire scenes. Although capturing in oblique isn't as scale friendly and as easily measurable as nadir it serves a purpose especially when mapping. With nadir there is not as much distortion and oblique can show far more detail such as height. To do modeling using images oblique is necessary yet time consuming.

Activity 4 - Gathering Ground Control Points

Introduction:
The primary focus on this activity was to get the class accustomed to using Ground Control Points or GCPs as a way to georeference and be able to geometrically correct aerial images taken by a UAS. To get the utmost accuracy one has to be very careful with the methods and equipment used in collecting GCP data. To prove this point we used a plethora of devices with GPS recording capability and compared them. When using GCPs in an area a bare minimum of three points has to be used. The more points you use, the better your end data will be. If you area has a lot of topographic variance more points will be needed to accurately gather data for that area. When focusing on an area GCPs should be well spread out and one should avoid placing them on the edge of the desired area because the further you get away from the center the more an image is distorted.

Study Area:
For this activity we moved to a new area (Figure 1) not as cut and dry as the soccer fields we are used to.

Figure 1 - Highlighted area where we placed our GCPs, located south of South Middle School

Weather conditions that day were adequate for the task at the hand, it was sunny and remained in the low 60s.

There was some varying terrain but nothing more than some tall grass and some cut trails. We focused on covering the area around the small pond. If there was more variance in elevation we would have had to of used more GCPs. 

Methods: 
The first step was laying down our GCPs in different locations, for this area we placed 6 around the pond in different spots in the study area shown in (Figure 1). You can use a number of different items as markers but for this exercise we were lucky enough to survey marker mats (Figure 2). 

Figure 2 - Marker we used as a GCP

These markers were nice because they were all uniform is size and gave us an exact cross to focus on in the middle. Consistency is key with GCPs to keep a high level of data integrity.

For data collection we used a wide variety of methods each with varying levels of accuracy. 

The first tool we used was a Dual Frequency Survey Grade GPS, the level of accuracy with this is unmatched but it comes with a price, 12k for educational use and 18k for commercial. (Figure 3)

Figure 3 - Dual Frequency Survey Grade GPS 

With all the devices we used it was very important we took the reading from the same spot on the GCP as you can see above (Figure 3) Ethan took his time making sure the device was level and in the center of the GCP.

The next two devices we used were similar to one another just had varying levels of preciseness and also different costs respectively. Both were Bad Elf GPS just one was of survey grade (Figure 4) at $600 while the other (Figure 5) was at the enthusiast level at $125. We paired both of these with a tablet to record the data.

Figure 4 - Survey Grade Bad Elf

Figure 5 - Bad Elf GPS Pro

The next device we used was a basic Garmin GPS unit, these can be purchased for less than $100. They do not claim to be pinpoint accurate but are more so aimed at giving you a general idea of where you are. Nonetheless we used this device for the sake of comparison. 

Also for the sake of comparison, we used a smartphone in two different ways to show how truly inaccurate cell phone GPS is and how it should not be used in surveying or with CGPs. We used ArcCollector and the GPS data associated with images taken on an iPhone. 

Lastly, we drew up a mission on Mission Planner (Figure 6)  and used the Matrix Quad Copter to do aerial imaging. This data will be processed and I will post the results after that is completed.

Figure 6 - Mission Plan we used for aerial imaging with our GCPs below.

Discussion/Results:

The devices we used all had their pros and cons. Generally the more expensive and complicated the better results you will get. The Dual Frequency Survey Grade GPS was cumbersome and expensive but highly accurate. The Bad Elf GPS units were easy to use and cheaper but you sacrifice some accuracy with those trade offs. The last devices really have no place in the GCP world but were interesting to use and compare to the more specified devices. Our results of the comparison can be found below (Figure 7). 

Figure 7 - GPS Unit Comparison

You may think by glancing at this that they were all pretty close but when it comes to surveying and GCPs we want pinpoint accuracy and that is where the expensive, harder to use equipment really shines through. 

Surveying technology, like any other field is rapidly growing and more portable options with precise accuracy will only become more and more available. Although the Dual Frequency Survey Grade GPS is amazingly accurate it is bulky, takes an extra amount of time to use and could be difficult to use with different terrains. The Bad Elf on the other hand is small, portable, much cheaper and still fairly accurate. 

Gathering GCPs is time consuming because one must be very precise with GCP location selection and collecting the data. Depending on the area being analyzed many factors such as topography, weather conditions, foot traffic, etc all have to be considered. 

This activity relates to commercial practices in a few ways but mostly in the sense that these are the tools professionals are using today and to be successful in this industry you not only have to know how to accurately collect the data but processing it correctly is equally as important. 

GCPs are important whenever accuracy is paramount. As our readings say, the more GCPs you have the better the quality of data but you must have over 3. 

GCPs relate to UAS missions because they are used as a second form of measuring an area. When processing the aerial images, GCPs will come in very handy for verifying the calculations are correct and the data is valid. 

Conclusion:

Although sometimes time consuming and not nearly as thrilling as flying a UAS to collect data GCPs play a vital role in GPS data collection and verifying the information you collect is correct and as accurate as possible. Depending on the device you use the level of accuracy is directly related. When using GCPs you must have at least three markers and if your area has varying levels of elevation that all has to be considered. As technology improves collecting GCP data will get quicker and easier while remaining very accurate.