Smart Farming with Automation – An Interwiev with Prof. Oksanen

Reisswolf: What sparked your interest in agricultural applications?

Prof. Oksanen: I was born on a farm in southern Finland, so I am a farm boy. After high school I went to the Technical University in Helsinki where I studied automation, control engineering and robotics. Initially, I had no plan to connect the studies with agriculture. After the Master’s degree came an opportunity to combine my knowledge of farming with my profession, automation. In the end, I did my PhD on the coverage path planning of agricultural field machinery. It involved computational geometry, robotics, and a lot of mechatronics to develop a theoretical approach to find a feasible and efficient path on the field, while taking into account practical constraints. The fact that I have been driving tractors since I was ten years old has definitely helped to understand the problems and develop the algorithms. I still enjoy doing farm work with tractors on the family farm occasionally.

Prof. Dr. -Ing. Timo Oksanen
https://www.amx.wzw.tum.de/
Number of employees: 7
Lecture offer:
Summer: Positioning and Navigation for Off-road Vehicles (Master module, 5 ECTS), Automated Agricultural Machines – Lab course (Practical course, 5 ECTS)
Winter: Tractor-Implement and Communication Technology (Master module, 5 ECTS), Tractor Engineering Fundamentals (Master module, 5 ECTS), Robotics and Automation in Agriculture (Bachelor and Master module, 3 ECTS)
Current research topics: Tractors and other agricultural vehicles; Tractor-Implement Automation
Communication technologies for vehicles, like ISOBUS; Autonomous agricultural vehicles / robots;
Navigation, guidance and planning; Model based control of mechatronic systems; Drives and power systems; Robotic implements
Contact person for theses and student assistant jobs: amx@wzw.tum.de

Reisswolf: The TUM had traditionally well-known chairs for agricultural applications, such as Prof. Renius at the Institute of Agricultural Machinery or Prof. Auernhammer from the Chair of Agricultural engineering. Is there a plan to build up on this basis or do you want to take the chair in new directions on agricultural applications in your research field?

Prof. Oksanen: That is right, Prof. Renius and Prof. Auernhammer were the world leading experts in agricultural mechanisation in the 1990s and early 2000s, so there remains a lot of legacy at the TUM. That is the reason why I was so interested in taking my current position, because there was the idea that I could try to combine these two research fields together at TUM and connect the campuses in a new way. My research portfolio includes topics similar to Prof. Renius’ research, but I am not yet an expert in his specifics such as gears and transmissions. Prof. Auernhammer was a leading contributor of electronics in agricultural machinery, especially to interchange information in tractor-implement systems. He was founder of LBS, the predecessor of ISOBUS, which is now the global standard for the tractor industry. I am currently contributing strongly for ISO standardisation just as he did in the 1980s and 1990s. I have known Prof. Auernhammer personally for 16 years – so it has been a smooth start in that field at TUM for me. Then of course I continue with the other research topics I have done in Finland, which are mainly related to automation, navigation, path planning and other kind of mechatronics when it comes to “intelligent machines in agriculture”.

Reisswolf: What lectures do you currently offer?

Prof. Oksanen: Tractor-Implement and Communication Technology was the very first course I established one year ago when I started. ISOBUS, the technology that is the industry standard in agriculture for tractor implement, is heavily based on the CAN bus technology. We also cover things such as GPS-data and practical aspects of agricultural machinery design. The technologies used in agricultural machines are similar to those available in automotive or even aviation. This course is included in ten different master programs, most of them are in mechanical engineering.

Our second course is Positioning and Navigation for Off-road Vehicles, which is also connected to similar master programs in mechanical engineering. We compared the technology for navigation and positioning for on- and off-road vehicles together with Prof. Lienkamp and found they are quite similar in their fundamentals. We thought that this is an interesting topic in general, but I wanted to highlight the off-road part in this course, since the vehicles for this terrain have slightly different requirements compared to on-road passenger cars. However, the content of this course has a close relation to autonomous vehicles in general. Whether on or off-road, they require positioning and navigation, even if they operate only semi-autonomously, such as recent tractors. It is interesting to look back and see that things like auto-steering / autopilot and so on have been available for farmers already 15 years ago. One major contributor to this technology development was certainly the introduction of GPS. In this course, the first seven or eight lectures contain the fundamentals of Global Navigation Satellite Systems (like GPS) positioning.  The second part of the lectures is related to navigation algorithms for robotics, such as in path planning and following, as well as local positioning sensors (LIDAR) or cooperative work with multiple vehicles. We initially planned two field days in our research facilities in Freising, so that students could collect their own datasets and get in touch with positioning hardware. Unfortunately, due to Covid-19 we had to cancel those plans in 2020. We hope to get back on the planned way in the next semesters.

Our third lecture is Tractor Engineering Fundamentals, originally from Prof Renius. Prof. Renius retired in 2003, when the Institute of Agricultural Machinery closed in Garching. He continued teaching this lecture voluntarily 17 years after his retirement. Now he is 82 years old and we have agreed that this winter semester I shall continue to teach the lectures myself. We have translated the course materials to English and increased the amount of ECTS to five due to the addition of exercises. We defined problems that students can solve at home after they got an introduction by the teaching assistant. The main tool for the exercises is Matlab, since it offers a good interface for programming and evaluating the fundamental equations of the course. For example, what are the forces related to ploughing or mechanisms from rear hitch. In the past, the course has been quite popular among students. This year, we had about fifteen students. The course is also at Master’s level and available in many study programs.

Our fourth course is Robotics and Automation in Agriculture from our team member Dr.-Ing. Ertug Olcay. We defined that it is possible for bachelor and master students and held in English. It is designed as a lecture only and thus rewarded with three ECTS. The course gives an overview of agricultural robotic applications, discusses subsystems of the robots and recent trends.

I decided to hold all my lectures at the campus in Garching

Reisswolf: Do you also plan to extend your lecture offer with a five ECTS bachelor course?

Prof. Oksanen: I do not have any plans to do that yet. The bachelor programs require coordinated planning with other professors, and we need to estimate what might be beneficial for the students in the future to further improve the bachelor programs.

One thing to add: Just before Covid started, I decided to hold all my lectures at the campus in Garching. During my first semester, I lectured in Weihenstephan only, hoping that students from Garching would come to Freising, but this was not the case. Now we hold all lectures at Garching, but every course comes with the extra spice of two field experiment days in Freising to get in touch with machines and smart farming equipment. This gives the students the opportunity to do field experiments and familiarise themselves with our research facilities. We have high-tech tractors, test fields, and other kind of equipment for mechatronic agriculture. I hope that we will have the opportunity to get back to that when Covid is finally over.

Reisswolf: Traditionally, Bavaria has a vibrant industry for agricultural engineering, such as AGCO/Fendt in Marktoberdorf, SDF Group in Lauingen, Ropa and so on. Are there plans to get in closer contact with those? The TUM is renowned to work very closely with the industry.

Prof. Oksanen: I have ongoing research projects with some of the companies that you mentioned. Traditionally strong collaboration with the industry was one of the reasons why I was excited to join the TUM. I have done this kind of collaboration in Finland with other companies for almost twenty years and have learned how to compromise and how to do engineering science with the industry. Since I am not new to the agricultural community, I also knew many of the German companies before I joined the TUM – mainly due to my affiliations to ISO standardisation and VDMA collaborations. When it comes to students and excursions, for instance in Tractor Engineering Fundamentals, I will continue Prof. Renius’ yearly tradition of doing a full day excursion to a company. The excursions in the past were mainly to the companies Fendt, Ropa or even to Lindner on the Austrian side. This year we organised a virtual excursion with Fendt and spent some time with very interesting discussions about engineering practices and future trends in the tractor market.

Reisswolf: Where do you see the future of farming in the next ten or twenty years? The trend goes to autonomously operated vehicles on the field while the driver can coordinate them from his office. The machines are growing in size. Will there be big, fully autonomous vehicles or rather several small independent robots working as a fleet like Fendt’s Xaver and MARS projects? Do you think that the situation will be the same on every continent? For example, how will Africa compare to North America?

Prof. Oksanen: Oh, that is a very good and very complicated question. The first thing that comes to my mind are the very large robot tractors, similar to what I have done in Finland about ten years ago. The idea of autonomous tractors has a long history, with patents starting from the 1800s. However, I cannot see that happening in the mass market for Europe over the next ten years. It might start in countries where it is hard to find labour to drive the tractors and typical fields are large and easy to operate. In Australia for instance, there is a low yield, a lot of acreage and farmers struggle to find capable employees to operate their machines. You are most probably aware that they also try to hire students from Germany for this job. There for example it might make economically sense.

The idea of autonomous tractors has a long history, with patents starting from the 1800s

There is a movement in Europe to make vehicles for special crops like vegetables or vine, the so-called high-prized crops. Another current trend is to develop some special vehicles for mechanical weeding and avoid using herbicides (like Glyphosate). When it comes to replacing these specialized machines with autonomous versions, it is again probably not going to happen within the next ten years in Europe. In the USA, in some cases the human drivers are significant contributors to the total production costs.  The fully autonomous vehicles are one option, but in many cases the economic viability favors highly automated rather than completely autonomous machines.

The MARS project and its successor Xaver of Fendt are in my opinion examples of farming robots that can become commercially available in the near future. The main difference to the versions mentioned earlier are the size and the task they can perform. Both are concepts of one-purpose-only vehicles, which in this case is precision planting. It can be compared to a milking robot, which has been quite popular on farms for the past twenty years – it can just do one thing which is milking cows. It became common due to its simplicity and the clear facilitation of the work. If you try to automate a tractor that does everything, it gets very difficult to make a generic and reliable system in all levels.
Looking at all the videos and pictures of fully autonomous tractors and combine harvesters in magazines and public media may seem contrary to my expected timeline.

I usually inform my students that they need to identify which department of a company has released some information of innovation. Was it the research and development department or the marketing department?

Was it the research and development department or the marketing department?

Reisswolf: Do you see cooperation between agricultural manufacturers and manufacturers of passenger vehicles when it comes to sophisticated automation and autonomy?

Prof. Oksanen: The key-components in the machines are nowadays closely related or often even similar. For instance, tractors and trucks share many components in the diesel engines, such as injection systems and exhaust gas management. Still, some differences are there due to varying requirements. Staying at the Diesel engine, the one in a truck can be long but not high, whereas it is the other way round in a standard tractor. As mentioned earlier, I try to cover this aspect in my lectures as well. It should however be mentioned that the tractor industry is usually one step behind the passenger cars. For instance, the very first vehicle to use the CAN-Bus technology was the Mercedes S-class in 1992. Maybe that is not surprising because the Mercedes S-class is high-end and trendsetter for innovations. The first time a tractor utilised CAN-Bus was in 1994, so the tractor industry followed the concept with a delay of just two years. The earth moving machinery comes a little bit later and, in many cases, they try to learn what the agricultural industry is doing, especially when it comes to ergonomic aspects and modularity. I have recently learned that there are surprising connections to aviation – tractors and helicopters have something in common, mainly the path planning and ergonomic aspects.

Reisswolf: What are limitations still present regarding machinery operation and its interaction with wildlife? How are autonomous vehicles currently dealing with it and do you think there is a development in this area?

Prof. Oksanen: We need to make autonomous vehicles robust, so they can tolerate these kinds of disturbances, which might be wildlife or just some wet spot where the vehicle could become stuck. This is one of the areas in which we are trying to do research: How can autonomous vehicles perform their work in challenging environments safely and reliably.  

Detecting different kinds of wildlife is one aspect. The other side of robustness is related to the mechatronic design of the machine, such as tolerance for extreme moisture, excessive crop residue, sensor signal faults or failures, vibrations, dirt and so on. We need to renew the thinking of designing implements and develop procedures to be independent of human-based rescue operations. If we think about ploughing, tillage, seed drills, baling and so on it is not enough to have sensors to detect a blockage. It is not helpful if the robot is simply calling the user saying “Hey, I am stuck, come and help me”. The vehicle also needs to be capable of retrieving operability on its own by cleaning itself. In the end, it is not only mechatronics and sensors – it might also require new kinds of materials and new designs for the basics of agricultural engineering that have been developing over the past 100 years.

The vehicle also needs to be capable of retrieving operability on its own by cleaning itself

That is what I am especially interested in: Finding practical solutions for robotic farming tools, one implement type at a time. Some are easier than others: For example, sprayers are not interacting with crops or soil directly and thus less prone to blockage. They still can get stuck on a wet spot, tip over on a steep slope or collide with obstacles in the field. When it comes to any implement that is interacting directly with soil or plants, there is always a risk that something can get blocked, which is particularly difficult to avoid.

Reisswolf: From my perspective, an additional challenge results from different automation levels for the typical actions a tractor operator does. What do you think needs to be done to get a large amount of implements to higher technology levels?

Prof. Oksanen: I think you are referring to the technology levels SAE defined for autonomous cars. I am not sure if it is interesting for the university to discuss those levels because that is more about the product for the end user. We try to develop methods, algorithms, principles and maybe some tools to improve the levels, but I do not have any special target where I want to be as this is up to the industry to find business roadmaps for their innovations. I just want to go systematically towards the higher levels, e.g. in algorithms, for it to be one day like SAE level 5. I just have the dream that autonomous vehicles can work without human interaction one full day. Unfortunately, I can only guess when this is going to be reality.

You have seen during the last twenty years which steps have been taken in that direction to introduce all kinds of technologies and subsystems. I do not only mean navigation via GPS but also tractor implement management, more automation in interconnected machines as well as wireless in-field communication so that the combine harvester and tractor can synchronize speeds for unloading. Moreover, there are automatic tire pressure control and other assistant systems that help so that the driver needs less expertise. In short: Automation in the vehicle helps the farmer to achieve the best performance.

Automation in the vehicle helps the farmer to achieve the best performance

Farming is consuming a large amount of fossil fuels and it will probably stay like that. It is currently very hard to replace a diesel engine with electric batteries due to the need of continuous high power output. On a shorter horizon, we need to improve the overall efficiency of agricultural production. A common scale is the produced amount of food per litre of diesel. The mission is to maximize this ratio with respect to every task involved on the farm, and maybe in the future we can keep the diesel equivalent in this equation just due to historical reasons.

Reisswolf: So at least in the medium term you do not see the agricultural machinery moving away from diesel engines.

Prof. Oksanen: There are certain operations where you can do that, maybe spraying is something where a special machine with batteries only is possible. However, when it comes to heavy pulling like ploughing or tillage, electrification with battery is not an option. The difference to passenger cars or trucks is that you are at full power almost all the time. However, trucks are only at full power when they accelerate, or they need to go uphill. It is a very small portion of the time and you can even harvest energy going downhill and charge the batteries. That is not the case when ploughing. Theoretically, batteries can of course be built to deliver the necessary output for the requested time – but it’s clear they would exceed acceptable frames for weight, price, and size remarkably. Recently, some concepts that use electric cables have been presented by a company, but I think that the diesel engine will stay for quite a long time as the main power unit for agriculture.

I think that the diesel engine will stay for quite a long time as the main power unit for agriculture

A saying goes: “the last drop of the diesel fuel on this planet will be used by a tractor”. I do not say that I totally agree with the saying, but it symbolizes the need for energy to produce food and we need to get it from somewhere.

Reisswolf: You already mentioned that there is a strong relationship between the automotive and truck technologies and the agricultural vehicles. Do you think that innovations in fuel cell technologies can also be part of the agricultural technology even in high power demands systems and that the engine, for example, will be replaced by a LNG engine or by a fuel cell system?

Prof. Oksanen: I think that LNG and biogases are quite close to the market. There have been prototypes by different companies, but the challenge is the distribution of these energies. The government wants to use hydrogen more, and I believe that the solution would be to be able to refuel trucks with hydrogen at every gas station. However, where does a farmer with a fuel cell tractor get fuel for that in the countryside? It is not only the vehicle but also the consideration of the logistics for the energy. I am very curious to see how trucks and heavy on-road vehicles will evolve in the next five years. If these logistical things can be improved, then the agricultural machinery businesses will also be interested.

Where does a farmer with a fuel cell tractor get fuel for that in the countryside?

Reisswolf: Sustainability and organic and vegan food are of growing importance. The agricultural industry has to listen to the consumer’s demands and follow the changing demand. What problems and challenges do new crops pose for agricultural technology? What future concepts are required with less livestock on the farms and more organic crops?

Prof. Oksanen: It would require a lecture series to answer such a difficult question. One idea I want to bring up is Pixelfarming, a concept introduced by a company in the Netherlands. This is one idea how technology could be involved in farming, a good example how new ideas can get possible with the help of robotics. I am not sure if I agree with it completely though. The idea is that each crop has its own 10x10cm “field” and robots can put different crops in every pixel, avoiding monocultures. The robots are driving in the field and cultivating the crops one at a time. The company has already some prototypes tested in the Netherlands. This is an interesting philosophy, enabled by robotics.

It is of course another situation in cities, where concepts of urban farming are great additions. People can grow crops at a small scale on limited space with different automation system like “vertical farming” or “plant factories”. However, this is not the scope of my research currently. Pixel farming is the closest one that I could imagine to your question.

Reisswolf: Do you have experience developing farming robots with students?

In Finland, we participated in a student competition called “Field Robot Event” for 14 years. It is an outdoor field competition for student teams where they have to build a small robot to solve four or five tasks. It is organised every summer somewhere in Europe. I have been in the steering committee of this competition. We started in 2005 and my last time was 2018. I have spent easily over 10.000 hours with students figuring out what the problems were with the robot prototypes. Teaching how to build a robot with students has been motivating.

For the competition last summer, we wanted to build a team from TUM, with students from mechanical engineering, informatics and life sciences, but we were unable to find enough students to form a team. We need at least eight students.

It is an event for automated robots, so any remote steering is strictly forbidden. The robot needs to have all necessary sensors and navigation skills. My student teams won the competition twice, with podium places many other times. I was quite excited to teach how to build the robot based on my experience and the fresh ideas from students with different backgrounds. At TUM there a lot a of student competitions such as student formulas and hyperloop, so I would like to find a team for 2022. We should start working in September 2021. We already have sponsors for this idea and all we require are interested students.

There could even be the possibility to host the competition in the next years. Information is available at fieldrobot.com.

Reisswolf: How about bigger field robots, such as autonomous tractor?

Here you are able to see the vehicle I have been working on in Finland since 2009. A start-up company built this tractor in 1989, which wanted to start production of robot-tractors for farmers. This is the only prototype they made with wheels, but they manufactured and sold seven or so tracked vehicles. It was a good exercise for me to make a complete control and navigation system, CAN-Bus, steering control and everything else required to operate this autonomous machine. The video is from 2012, we had 25 hectares and we did different experiments with the so called Apu-Module. However, it is an old prototype machine, 31 years old, so it constantly required maintenance and suffered lots of break downs.

We also did fertiliser spreading and harrowing with this machine: This was the most recent project in Finland, autonomous tractors harrowing and sowing simultaneously. The challenge was to do the mission planning where two vehicles need to be routed optimally. Firstly, you need to harrow and subsequently continue with the seeding robot. The tractors are coordinated in such a way that they are in the right place at the right time to avoid collisions and save fuel. It is particularly challenging to react to disturbances in real-time and adjust to the plans accordingly. If you watch the video carefully, you can see me standing in the middle of the field, operating the videodrone.

Reisswolf: Thank you very much for this interesting interview!