تدريب Shadowing: Controlled Multi-modal Image Generation for Plant Growth Modeling - تعلم التحدث بالإنجليزية عبر الفيديو

Hello my name is Libana Roscher

and I want to present our paper Controlled multimodal image generation for plant growth modeling

which was presented at the international conference on pattern recognition last summer.

So when we have a look at such an aquaculture field

and we work with robotics and with machine learning methods for phenotyping

we can ask ourselves different questions

so for example we can ask what is on the field

which is a typical classification task

or we can ask what is the yield in a specific area

which is a typical regression task

or we can ask ourselves what will a plant look like in the future

and this is a forecasting tasks

and this is exactly the question we addressed in our paper

so imagine the following scenario you already know that

when plants grow closely together this can be very beneficial

so for example it can increase the biomass the crucial question is now what is actually a good mixture

and so what you can do is you can test different mixtures then you can collect data

and then you can learn a model from this information

and this model relates the information from what we have measured to a parameter like the biomass

so you have a relationship between the mixtures and the biomass

and then the good thing is this model can be applied to new mixtures

and these new mixtures were not tested yet in the field before

so you get an idea what potential good mixtures are

And one problem is now that one might be interested in the biomass

and the other might be interested in the spread of diseases.

So it would be good to combine this and to get complete information.

And this can be done if you model the whole appearance with plants, that means if you generate images.

And with this you are flexible in what you do with the result

because it can be treated such as an artificial sensor measurement.

So in order to generate such images we used

so -called generative adversarial networks illustrated here and this is a neural network consisting of two parts.

So one part is the generator G and this generator is responsible to generate images.

The other part is the discriminator, and the discriminator helps to train the generator to generate really good images.

The discriminator does it by trying to distinguish between real images and images which were generated by the generator.

So in the best case, the generator generates such good images

that the discriminator can no longer distinguish between a real image

and an image which was generated but there's one crucial thing

which is important for our application

so we do not want only want to generate an arbitrary

nice -looking image we want to condition on specific factors on specific things

and one important thing is we want to condition on the appearance of an of an additional input image

which is from an earlier point in time

and the second important thing is we want to condition on

so -called factors of variation and these factors of variation are factors

which influence the appearance of the of the plants

and the novelty of our paper is exactly this part

because on the one hand we learned the representation

which is useful for the machine learning model but on the other side it was

Because we can also easily and intuitively change these factors of variation so that we can produce different scenarios, different outcomes.

And so here you can see results of

which we presented in our paper where we generated images with chain factors of variation.

So our dataset contains UAV images, which were captures in Kampus -Klein -Altendorf,

where we have different mixtures of spring wheat and faba bean.

And we also collected different factors of variation, which belong to these mixtures.

And they are illustrated in green and light blue and in dark blue.

And on the left, you can see the input image, which is from an early growth stage.

And this is the real scene from a later growth stage and the measured factors of variation.

And these images are all generated images.

So here we change the factors of variation.

So for example, on the very right image, you can see we set all factors of variation for Faba bean to zero.

And actually there are no Faba bean plants in the generated image.

So we can clearly control and manipulate how the images can be generated it

and from these images we can now derive specific parameters pointing

us to potential promising mixtures we can later test in the

field with this thank you very much for listening you can find the paper by scanning the qr code

and if you are more interested in this research direction you can go to our website

or you can have a look in my YouTube channel.

Thanks a lot!

ما هي تقنية التظليل الصوتي؟

التظليل الصوتي (Shadowing) تقنية تعلم لغة مدعومة علمياً، طُورت أصلاً لتدريب المترجمين الفوريين المحترفين. الطريقة بسيطة لكنها قوية: تستمع لصوت إنجليزي أصلي وتكرره فوراً بصوت عالٍ — كظل يتبع المتحدث بتأخير 1-2 ثانية. تُظهر الأبحاث تحسناً كبيراً في دقة النطق والتنغيم والإيقاع وربط الأصوات والاستماع والطلاقة.

تدريب Shadowing: Controlled Multi-modal Image Generation for Plant Growth Modeling - تعلم التحدث بالإنجليزية عبر الفيديو

السيناريو

المجموعات المفيدة والتعابير الاصطلاحية

تحدي التظليل الخاص بك

ما هي تقنية التظليل الصوتي؟