"Integration of AI solutions in production" course
40 recorded sessions / certificate
starts June 22 / price - 39.900 rubles
Apply for course
starts June 22 / price - 39.900 rubles
Apply for course
By clicking the button, I consent to the processing of personal data and agree to the privacy policy and the offer agreement.
"Integration of AI solutions in production" course
40 recorded sessions / certificate
starts June 22 / price - 39.900 rubles
Apply for course
starts June 22 / price - 39.900 rubles
Apply for course
By clicking the button, I consent to the processing of personal data and agree to the privacy policy and the offer agreement.
Course Objectives
Learn to integrate solutions based on neural networks and machine learning into production systems of any complexity
- 1Master modern methods and frameworks for working with pre-trained neural networks.If you already use frameworks and modern components to work with neural networks, the course will help to expand your horizons in this area.
- 2Gain skills in Full Stack ML development.If you have not previously worked with specialized neural network frameworks, you can quickly and deeply explore this area and start using them in the middle of your course.
- 3Examine the main components of data processing and preparation (Data Engendering).The methods of data processing and Data Engendering will be useful not only for neural networks, but also will help to transfer data between different systems efficiently and reliably, to set up and manage data loading and processing.
- 4Learn to quickly integrate a trained neural network into a wide range of industrial systems.The knowledge gained can be used not only for the integration of neural networks and various industrial systems, but also for the integration of the two systems with each other or for the integration of industrial systems and Python-programs.
- 5Learn to set tasks for the development team (including those who are not familiar with neutron networks) to transfer the trained models to production.You will learn to speak the language of the developers
- 6Learn to "pack" the trained neural network in a format convenient for the end user (Web-services, mobile applications, IoT devices, etc.).In today's world, startups consisting of 2-3 programmers are able to present a commercially ready product; upon completion of the course you will be able to perform tasks of a small team of developers.
The aim of the course is to give participants skills in Full Stack ML development. Upon completion of the course, participants will be able to create complex Pipeline data processing, independently deploy Data engeneering components, set tasks for the development team (to transfer the model to production), independently prepare the model for output to production (deploy all necessary components and package the model in a format suitable for transfer).
The course is designed for participants who are familiar with the basic concepts of neural networks, but have difficulties with the practical application of a neural network in real systems/tasks, and aims to give practical knowledge, as the already trained neural network can be applied in real tasks from A to Z (excluding the nuances related directly to the training of the neural network).Курс рассчитан на участников, знакомых с базовыми понятиями нейронных сетей, но имеющих трудности с практическим применением нейронной сети в реальных системах/задачах, и нацелен на то, чтобы дать практические знания, как уже обученную нейронную сеть можно применить в реальных задачах от А до Я (исключая нюансы, связанные непосредственно с обучением нейронной сети).
The course covers all steps: how and where to take data for an already trained neural network, how to properly feed it into the network, how to extract the results of predictions, how to give these results to different systems, how to make the neural network available online (in the cloud) and much more.
Training Format
- 40 recorded sessionsYou will have 40 webinar records available for viewing at any time
- 40 practical tasksAfter each lesson you will get a practical task
- 9 months of curator supportDuring the week, you will be able to ask any questions about how to perform tasks to your tutor in the common chat
What are you going to learn from the course
Web-crawling
(parsing and uploading data from sites, analyzing APIs)
Methods of data storage and presentation in relational and non-relational databases (DBs)
Relational data storage units
Postgres / Redshift / Oracle
MPP storages
(Teradata, SAP HANA)
Storages of unstructured data
(Hadoop, AWS S3)
Key-Value data warehouses
(Hbase, MongoDB)
In-Memory systems
Apache Solr, GPU-based databases
Real-Time data delivery systems
Apache Kafka, Spark Streaming
The most powerful tool for handling large amounts of data
Apache Spark
One of the most advanced frameworks for working with trained models
TensorFLow - Tensor Flow Extended
Containerization of applications in Docker/Kubernetis
Powerful AWS (Amazon Web Services) based development toolbox
API Gateway/AWS Lambda/AWS Fargate
Book a discount and a course as a gift
Apply and our manager will call you back
By clicking the button, I agree to the privacy policy
Course teacher and methodologist
Gerard Kostin
31 y.o., Innsbruck, Austria
Extensive experience in solving problems using ML methods in the following areas: telecommunications, banking sector, Internet providers, industrial enterprises, e-commerce.
Practical experience of work with big data (clusters from 5 Pb). During his career in the field of Data Science he launched these areas from scratch in companies such as MTS and Rostelecom, worked together with companies from the U.S. silicon valley and EU companies.
Practical experience of work with big data (clusters from 5 Pb). During his career in the field of Data Science he launched these areas from scratch in companies such as MTS and Rostelecom, worked together with companies from the U.S. silicon valley and EU companies.
Training program
Part #1. Theory
19 1-hour lectures
Data, their types and source systems
(5 sessions)
(5 sessions)
Description: The first lesson begins with the most important topic - data; any unclosed program (and especially neural networks) requires data delivery for its work. In this lesson we will get acquainted with the main types of data sources.
Assignment: A logical task on the topic of the lesson.
Assignment: A logical task on the topic of the lesson.
Description: in the second lesson we will take a closer look at the features and options of data storage in source systems and understand the basic terms.
Assignment: A logical task on the topic of the lesson.
Assignment: A logical task on the topic of the lesson.
Description: In the lesson we will learn what types of data storage are, understand the advantages and disadvantages of the most popular types of data storage.
Assignment: The task is to select the optimal type of data storage.
Assignment: The task is to select the optimal type of data storage.
Description: The fourth lesson reveals details about the ways and methods of obtaining data from source systems. We will examine the most popular data collection methods and tools and learn how to do this using Python.
Task: Practical task for exporting data from source systems
Task: Practical task for exporting data from source systems
Description: Modern Internet is the largest source of information, in the class we will understand how to download data from Internet sites, how to explore sites for a convenient API to access the data, how to emulate the work of the browser and how to automate this process with the help of RootShop.
Task: Web-crawling of real sites
Task: Web-crawling of real sites
Data model (5 sessions)
Description: In the class we will analyze in detail the "golden standard" of data representation in modern structured data warehouses: 2nd and 3rd normal forms.
Assignment: Practical assignment for presenting data in the 2nd and 3rd normal forms.
Assignment: Practical assignment for presenting data in the 2nd and 3rd normal forms.
Description: The key-value type data storage approach is used in most modern systems where the speed of information output by key is important. In the lesson we will learn in detail about this architecture and the benefits it provides.
Assignment: A practical task of converting unstructured data to Key-Value type.
Assignment: A practical task of converting unstructured data to Key-Value type.
Description: When there is a lot of different data in the storage, it becomes difficult to navigate how and in what form the data is stored. In this class, we will understand the concept of data schema, learn how it helps to navigate a storage, and learn how to create data schemas.
Task: Practical task for creating data schematics
Task: Practical task for creating data schematics
Description: in any large organization there are hundreds and thousands of data tables in storage, not all of these data are useful and necessary for analysts and machine learning models. Datamarts allow you to organize data in such a way as to get access only to the information and only in the form in which it is necessary, excluding unnecessary. In the lesson we will understand how to organize data in datamarts and work with them.
Task: practical task for working with datamarts
Task: practical task for working with datamarts
Description: A large structured repository consists of hundreds of tables linked to each other, the visualization and management of these links are performed by data modeling tools. In the lesson we will get acquainted with the most popular commercial and non-commercial data modeling solutions.
Assignment: A practical task for building a data model.
Assignment: A practical task for building a data model.
ETL/ELT (5 sessions)
Description: In this lesson we will learn the most common ways to upload data (obtained from a user or machine learning model) to various data sources.
Assignment: practical task of uploading data to a data source system.
Assignment: practical task of uploading data to a data source system.
Description: the most popular paradigm of data transfer between different systems is ETL (Extract Transform Load). Often data needs to be transformed in one way or another before being uploaded to the storage, the ETL paradigm implies that data is extracted "as is", then transformed and only then delivered to the storage. In the lesson we will get acquainted with this process in detail.
Assignment: practical task on the topic of the lesson
Assignment: practical task on the topic of the lesson
Description: The session is dedicated to a very important topic - data integrity and quality. Incorrect or corrupted data can lead to serious consequences, especially if the problem is not immediately identified. We will learn basic methods and tools to control data integrity and quality.
Assignment: practical task on a lesson topic.
Assignment: practical task on a lesson topic.
Description: If an error occurs during data upload, it may cause business processes to stop or data loss. In the lesson we will learn the basic methods and systems for monitoring ETL/ELT processes.
Assignment: practical lesson topic assignment
Assignment: practical lesson topic assignment
Description: Due to its simplicity and clarity, the ETL paradigm can be implemented with the simplest means (e.g. a set of scripts), but when there are a lot of ETL processes and they are subject to change, one cannot do without a powerful system to manage these processes. In the lesson we will learn several modern frameworks for managing ETL/ELT
Assignment: practical assignment on the session topic
Assignment: practical assignment on the session topic
Transferring the model between languages (4 sessions)
Description: If the model is written in one language (for example, Python), and should be executed in an industrial system in a completely different language and interface, there is a problem of transfer and compatibility model. In this session, we will get acquainted with the universal descriptive language model PMML and understand how it is possible to transfer the model from one language to another (with support of the PMML-standard).
Task: practical task of transferring the trained model to the PMML standard.
Task: practical task of transferring the trained model to the PMML standard.
Description: The universal PMML standard is not the only tool to transfer the model between programming languages and different systems. In the lesson we will learn about different variants of cross-platform model transfer from Python to C.
Assignment: practical task of transferring the trained model from Python to C.
Assignment: practical task of transferring the trained model from Python to C.
Description: In the class we will touch upon the most frequently encountered problems when porting models using the example of the Keras C model port and analyze the solution options.
Assignment: practical assignment on the session topic
Assignment: practical assignment on the session topic
Description: Containerization technology is a kind of compact virtual machine that contains only the necessary components to run your application. In the lesson we will learn in detail about containerization technologies and the advantages they provide.
Task: A practical task on creating a simple container.
Task: A practical task on creating a simple container.
Part #2. Data sources
7 30-minute sessions
Description: in the session we will analyze in detail the architecture and features of the three most popular relational databases Postgres / Redshift / Oracle and learn to deploy your own database.
Task: practical task of creating a database and its initial configuration
Task: practical task of creating a database and its initial configuration
Description: this lesson is devoted to analytical storages (optimized for data requests), we will consider the general architecture of such storages MPP (Massive Parallel Processing) and learn what are the features of working with these storages
Assignment: practical task - working with test repositories of MPP architecture
Assignment: practical task - working with test repositories of MPP architecture
Description: In the era of big data, it became possible to collect, store and analyze petabyte data volumes. Most of the information in such storages is unstructured (equipment logs, texts, video/photo stream, detailed application logs). It is impossible to work with such data by traditional methods. In this lesson we will analyze the revolutionary technologies: Apache Hadoop and AWS S3, which made effective work with petabytes of data possible.
Task: practical task on starting a virtual machine with Hadoop cluster and working with it, task on AWS S3.
Task: practical task on starting a virtual machine with Hadoop cluster and working with it, task on AWS S3.
Description: Key-value specialized storages compatible with large data technologies is the topic of this class.
Assignment: practical task on working with Key-Value repositories.
Assignment: practical task on working with Key-Value repositories.
Description: The speed of computing and data transformation is a key factor for many modern applications and systems. In the class we will consider advanced architectures that allow to calculate, store data in memory and perform calculations on the fly, as well as pay attention to innovative databases based on the GPU (graphics processing unit) systems.
Assignment: practical task for working with in-memory systems
Assignment: practical task for working with in-memory systems
Description: In the class we systematize knowledge on how to effectively access data stored in different sources with the help of RootSon programming language.
Task: a practical task to obtain data from various sources when using PTT.
Task: a practical task to obtain data from various sources when using PTT.
Description: The lesson will focus on a detailed overview of methods and techniques of data delivery and processing in real time. A tool such as Apache Kafka allows you to effectively configure the transmission of streaming data (data coming directly from source systems) to different consumers: repositories and programs.
Task: practical task on setting up the Apache Kafka
Task: practical task on setting up the Apache Kafka
Part #3. Practice
14 sessions of 30 minutes - 2 hours
Integration into production (14 sessions).
Sessions in this part are 100% practical, we will install, customize and combine various components to bring models and machine learning solutions into production. In the lessons there will be a minimum of theory, we will solve real cases from practice, there will be a lot of work by hand, we will write code. During the process, the teacher will share with you the most valuable practical experience and advice.
Sessions in this part are 100% practical, we will install, customize and combine various components to bring models and machine learning solutions into production. In the lessons there will be a minimum of theory, we will solve real cases from practice, there will be a lot of work by hand, we will write code. During the process, the teacher will share with you the most valuable practical experience and advice.
Description: from the first lesson, we will deploy a working storage based on the Hadoop cluster and solve a small practical problem, armed with theoretical knowledge gained in the previous sections.
Task: practical case (development, deployment and configuration of the storage based on the task).
Task: practical case (development, deployment and configuration of the storage based on the task).
Description: we will continue to work with the cluster created in the first lesson, we will configure the uploading of data streams to the cluster using compatible ETL tools.
Task: practical case (submitting data streams to your storage from RS #1).
Task: practical case (submitting data streams to your storage from RS #1).
Description: Cloud storage systems are the most powerful tool of Data Science engineer, significantly saving the labor costs of working with data warehouses. In this lesson we will learn to work with AWS cloud platform, we will deploy AWS Redshift database and create AWS S3 object storage. Using this infrastructure, we will try to configure ETL to integrate these systems with the cluster from previous lessons.
Task: practical case (feeding data streams to your storage from RS #1).
Task: practical case (feeding data streams to your storage from RS #1).
Description: In the session we will learn in detail about the features of Apache Spark framework for handling large data stored in hasoop. We will learn how to transform data and conduct data research using Spark. There will be a lot of coding and practical tips.
Task: A practical case (data transformation, data analysis).
Task: A practical case (data transformation, data analysis).
Description: We will continue to explore the capabilities of the powerful Apache Spark framework, this time we will consider methods and functions for solving machine learning problems when working with large data and learn how to build complex data processing chains, followed by automated application of machine learning models.
Task: practical case (setting up a data processing chain with machine learning model included)
Task: practical case (setting up a data processing chain with machine learning model included)
Description: In addition to the classic machine learning methods, Apache Spark allows you to work with neural networks. We will parse the nuances of work and learn to run trained neural networks using Spark.
Task: A practical case (using pre-trained networks based on Apache Spark).
Task: A practical case (using pre-trained networks based on Apache Spark).
Description: This session opens a new topic - using Tensor Flow Extended framework to automate the launch and application of trained neural networks in practical tasks.
Assignment: practical case (automating the launch of the trained model using tf serving).
Assignment: practical case (automating the launch of the trained model using tf serving).
Description: Expanding knowledge of Tensor Flow Extended framework, we will consider its capabilities to automatically process and prepare data for trained neural networks.
Assignment: practical case (data preparation and preprocessing).
Assignment: practical case (data preparation and preprocessing).
Description: the lesson opens a new block of lessons on application containerization. Using theoretical knowledge from part 1, we will learn how to containerize trained machine learning models with all dependencies.
Assignment: practical case (preparation and packing into containers of machine learning models with all the components).
Assignment: practical case (preparation and packing into containers of machine learning models with all the components).
Description: we continue to delve into containerization, this session is dedicated to studying the Kubernetis framework. The framework allows you to flexibly manage Docker containers, manage resources in containers and much more.
Task: practical case (organize management and resource allocation between a group of Docker containers).
Task: practical case (organize management and resource allocation between a group of Docker containers).
Description: Another topic related to the containerisation of the Google framework, which aims to automate all processes for creating machine learning models from A to Z, will be covered in the lesson.
Assignment: practical case (working with the framework)
Assignment: practical case (working with the framework)
Description: In this lesson we will continue to delve deeper into cloud computing capabilities. In this lesson we will parse the AWS API Gateway functionality and use it to make MOCK UP ("imitation") of Real-Time functionality to obtain data from the data warehouse via REST-interface.
Assignment: practical case (working with Gateway API)
Assignment: practical case (working with Gateway API)
Description: The lesson topic will reveal the incredible potential of AWS Lambda (mini-functions written in various programming languages) as a computational tool in conjunction with the Gateway API. In the lesson, using Lambda (in Python), we will write a working industrial solution that makes it possible to output data from our cluster in real-time mode by REST request.
Task: practical case (integration of AWS Lambda with Gateway API and data sources).
Task: practical case (integration of AWS Lambda with Gateway API and data sources).
Description: Another session that covers cloud computing and application containerization. In this session we will learn about AWS Fargate's capabilities to run and manage containers in the cloud. We will write a simple web interactive application that runs a trained model, pack it in a container and make the container available online.
Task: practical case (launch your own containers based on AWS Fargate).
Task: practical case (launch your own containers based on AWS Fargate).
Apply for course
Our manager will answer any questions about the course
- Artem Voronov-GashevCustomer service managerEmail: info@neural-university.ru
Phone: +7 (499) 648-67-44
Whatsapp/viber/telegram:
+7(918) 916-41-84
"Integration of AI solutions in production" course
40 recorded sessions / certificate
starts June 22 / price - 39.900 rubles
Apply for course
starts June 22 / price - 39.900 rubles
Apply for course
By clicking the button, I consent to the processing of personal data and agree to the privacy policy and the offer agreement.
"Integration of AI solutions in production" course
40 recorded sessions / certificate
starts June 22 / price - 39.900 rubles
Apply for course
starts June 22 / price - 39.900 rubles
Apply for course
By clicking the button, I consent to the processing of personal data and agree to the privacy policy and the offer agreement.