Recurrent neural networksRecurrent neural networksRecurrent neural networksRecurrent neural networks
  • Products
  • How Dature works
  • About us
  • Knowledge base
  • Partnership
  • Support
  • UE Flaga
  • English
    • Polish
English
  • Polish
✕
            Brak wyników wyszukiwania Zobacz wszystkie wyniki
            How the AI revolution is changing business
            19 January 2022
            Prediction intervals as quantification of predictions using a neural network
            15 February 2022
            Updates

            Recurrent neural networks

            Recursive Neural Networks (RNNs) are used to model and predict variables that evolve over time. Unlike standard neural networks, they have the ability to remember the context they are in. A traditional neural network assumes that all inputs (and outputs) are independent of each other, while RNNs perform the same task for each element of the sequence, and the output depends on previous calculations. Many practical problems, such as recognition and processing of text, speech, images, are solved more efficiently if they take into account previous states of variables. It is this dependence of the current state on previous states that recurrent networks attempt to reproduce. For this purpose, loops are introduced into their structure. Thus, certain parts of the network are repeated. In other words, RNNs contain feedback, that is, connections that provide output signals located in the downstream layers of the neural network to neurons located in the input layer or in the closer hidden layers. This makes it possible to perform more complex calculations than networks with only unidirectional information transfer.

            The recursiveness of RNNs boils down to performing the same task for each element of the sequence. Intuition aside, one can think of an RNN as a being that has a memory storing information about past calculations.

            Figure 1. Hopfield neural network diagram

            Source: Basic neural network models
            http://books.icse.us.edu.pl/runestone/static/ai/SztuczneSieciNeuronowe/PodstawoweModeleSieciNeuronowych.html

            The RNN concept is not new, the recursive architecture was already proposed in the 1980s by Hopfield (a schematic of its design is shown in Figure 1). The structure of a Hopfield network can be described as an arrangement of many identical elements connected to each other on a one-to-one basis. The introduction of feedback loops results in multiple excitation of neurons in a single cycle. Hopfield’s work contributed to the revival of neural networks research, which, after a period of tumultuous development in the 1950s and 1960s, was abruptly halted in the early 1970s. The aforementioned resurgence involved accelerated research and development of RNNs.

            It turned out that while RNNs have memory, it is a rather short memory. Theoretically, neurons with recurrent connections are potentially able to model relationships occurring with arbitrarily long intervals, but practice does not confirm this. The RNN’s short memory is the result of the gradual fading of the first inputs, with the result that, after a certain time, the RNN state contains virtually no traces of the initial inputs. From a formal point of view, the short memory of RNNs is associated with a fading gradient for time-distant relationships, with the fading occurring exponentially with time. This is not the only problem, as RNNs often struggle with learning instability, often equated with overtraining, and also referred to as an exploding gradient. The latter problem can be solved by using gradient pruning, which is the restriction of gradient values, and regularization, which is the penalty for over-complexity of the estimated model. The problem of gradient fading, on the other hand, is generally solved by introducing different types of cells with long-term memory.

            The Long short-term memory (LSTM) network is based on just such a solution. They were proposed by Hochreiter and Schimdhuber in 1997. In addition to standard units, the LSTM also has three types of special gates to control the stored data (Figure 2):

            • input gate, finding which value from the input to use to modify the memory.
            • forgetting gate, whose task is to determine what details should be removed from the block,
            • The output gate controls which parts of the long-term state should be read and output at any given time.

            Figure 2. LSTM neuron architecture

            Source: https://colah.github.io/posts/2015-08-Understanding-LSTMs/

            The addition of gates has significantly improved the efficiency of recurrent networks. RNNs with LSTM units allowed, among other things:

            • Facebook’s automation of translations (FB performs several billion such translations daily),
            • Training a neural network that controls a human-like robotic hand that manipulates objects with high precision,
            • Writing an AlphaStar program to compete in the complex real-time strategy game StarCraft II.

            Areas of further application are limited only by human imagination.

            This article was written thanks to the funds from the European Union’s co-financing of the Operational Program Intelligent Development 2014-2020, a project implemented under the competition of the National Center for Research and Development: under the “Fast Track” competition for micro, small and medium-sized entrepreneurs – competition for projects from less developed regions under Measure 1.1: R&D projects of enterprises Sub-measure 1.1.1 Industrial research and development work carried out by enterprises. Project title: “Developing software to improve forecast accuracy and inventory optimization from the perspective of customer and supplier collaborating in the supply chain using fuzzy deep neural networks.

            Udostępnij
            LAST UPDATES
            • Service level indicators in inventory management - how to understand and interpret them?
            • The essence of the classic model of inventory renewal based on the information level - the point of reordering
            • Fuzzy systems
            • The benefits of using artificial intelligence in the supply chain
            TAGS
            • #bullwhip-effect
            • #covid19
            • #demand-forecasting
            • #forecasting
            • #Intelligent-Development-Operational-Program-2014-2020.
            • #inventory-management
            • #inventory-optimization
            • #NCBiR
            • #neural-networks
            • #out-of-stock
            • #outllier
            • #overstock
            • #safety-stock
            • #safety-stock
            • #seasonal-stock
            • #service-level-suppliers
            • #stock-projection
            • #stock-projection-over-time
            • #supply-chain
            • #supplychain

            Related entries

            29 July 2022

            The essence of the classic model of inventory renewal based on the information level – the point of reordering


            READ

            The main feature of the model based on the so-called. “ordering point,” also known as an information-level ordering system or continuous review [3], is a condition […]

            15 July 2022

            Fuzzy systems


            READ

            Based on fuzzy sets, a fuzzy inference system can be built. In such a system, fuzzy rules are implemented for modeling, which in turn make it possible to carry out the process of fuzzy inference.

            15 July 2022

            The benefits of using artificial intelligence in the supply chain


            READ

            According to Gartner, the level of automation in supply chain processes will double in the next five years. At the same time, global spending on IIoT […]

            Right now

            Subscribe to our newsletter

              Providing data in the form is necessary to subscribe to the newsletter. Data from the form will be processed on the basis of the consent given Art. 6 paragraph. 1(a) RODO. The administrator of your data is Smartstock Sp. z o.o. from Poznan (ul. Królowej Jadwigi 43, 61-871 Poznań, biuro@smartstock.cloud). You can find all information regarding the processing of the data provided in the form and your rights in the Privacy Policy.


              TEL. +48 534 288 279
              biuro@dature.cloud

              Smartstock Sp z o.o.
              ul. Królowej Jadwigi 43
              61-871 Poznań

              Subscribe to our newsletter

              NIP: 7812000413
              REGON: 384284847
              KRS: 0000802331

              • LinkedIn
              • Facebook

              Privacy and cookies policy   Terms and conditions of the application   Technical support   Change in the decision on cookies

              All rights reserved. © SMARTSTOCK Sp. z o.o.
              English
                        Brak wyników wyszukiwania Zobacz wszystkie wyniki
                        • Polish
                        • English