A new commentary and research study by conservation biologist, Zhengyang Wang, has revealed that gaming engines, such as the popular Unity and Unreal Engine, can in fact be used to render hyper-realistic wildlife and animal images that are good enough to pass as legit images when shown to specific AI software.
These learning AIs use images – including these gaming engine-generated ones – as a basis for their training to recognize animals that might pass by and trigger a camera trap. Having extra images helps gradually improve the quality of what is referred to as rare-class categorization.
• These are the best trail cameras in 2023
Plenty of wildlife and nature photographers will have at some point used or come across camera traps, also known as trail cameras. The purpose of these cameras is to be subtle, often camouflaged to match a certain surrounding. They can be remotely triggered to capture images of animals, or whatever else that might pass by the sensors, in isolated areas.
Trail cameras are usually fitted with motion detectors, as well as night vision and infrared cameras, and sometimes even thermal imaging cameras. Animals that are naturally shy won't usually approach a camera trap, but can still be monitored by using this equipment intended to detect signs of life and movement.
Zhengyang Wang, a conservation biologist, has shed some light on a known problem of trail cameras and identified a unique way in which the puzzle could be solved. Identifying common wildlife has become easy for most modern camera trap AI. Spotting rare animals, however, remains to be a struggle.
In a recent commentary shared to Mongabay, Wang proposes that computer-generated images of animals that have been created by popular gaming software engines could be used to train camera trap AIs to detect rarer species that it otherwise wouldn't recognize without previous imagery. Images generated this way should not be confused with AI; this is the process of 3D graphics, so images would come from a digital model based on a real animal.
"If a species is rarely seen in camera trap photos, there isn’t enough data for it to learn, and it won’t be very good at recognizing that rare species" Shares Wang. "AI is able to learn from the kind of game engine-generated, hyper-realistic animal images that feature in today’s highly advanced digital games."
Advancements in rendering capabilities of modern game engines have meant that virtual worlds, especially open-world games, are becoming more realistic and immersive by the second. This is why plenty of virtual and in-game photographers are beginning to make a living and earn employment for their digital services.
• These are the most picturesque videogames for virtual photographers
Wang observed during a chance meeting with some game studio developers, or as he calls them "a rakish horde of digital nomads" in a rainforest in Panama that these tools used by game developers are actually helping real-world conservationists, even if they appear to be cherishing biodiversity "for an alien reason" as Wang puts it.
Putting the Unity theory into practice
The bottom line is that conservation practitioners who use multiple camera traps for extensive mammal surveys have the painstaking and time-consuming job of identifying the animals captured in each shot. With the help of AI, this process is made much easier, although before an AI can tell the difference between a deer and a badger, it must be trained by annotated images.
Wang explains that this is the problem of rare-class categorization, whereby computer systems struggle to categorize objects that are rarely seen during training, such as sightings of rare and endangered species, and collecting additional training examples is often an expensive feat.
Luckily a colleague of Wang's, Sara Beery, and her team was able to put this theory into practice during an experiment to increase her AI’s ability to detect deer. They used developers to create images in a simulated forest with digital deer using the Unity game engine.
These virtual photos of deer were fed to AI, and its deer detection rate increased by 40%. The study, "Synthetic Examples Improve Generalization for Rare Classes,” was authored by Sara in May 2019 and explores different simulation methods and their effects on classification performance.
Will conservation become reliant on gaming tech and hyper-realistic models of nature? Let us know what you think!
• You may also be interested in the best portable hides and camouflage gear for wildlife photography, as well as the best cameras for wildlife photography, and not forgetting the best lenses for landscapes too.
• Check out these 10 ultimate locations for wildlife photographers, and the 10 best games for virtual photographers.
