Trainable entropy system

Claim 2

A trainable system of cascaded processors comprised of: a plurality of non-linear signal processors in cascade, each processor including: a. means for applying at least one input signal provided by peripheral equipment and one probabilistic signal generated by the previous processor in the casecade thereto; b. means for applying at least one desired output signal provided by peripheral equipment thereto when such processor is operated in a training mode; c. a multi-level tree-arranged storage array having at least a root level and a leaf level; d. means for defining a path through the levels of the tree from said input and probabilistic signals; said leaf level including means for accumulating and storing the number of occurrences that each desired output signal was associated with each of said defined paths during training; and e. means for generating at least one output signal comprising the probabilistic signal for the subsequent processor in the cascade when the processor is operated in an execution mode.

Claim 3

The system of claim 1 wherein said probabilistic signal for the subsequent processor in the cascade is generated from said accumulated and stored number of occurrences when the processor is operated in an execution mode.

Claim 4

The system of claim 2 including preprocessor means for encoding said at least one input signal into one or more key components, and each of said processors including: means for encoding said probabilistic signal into one or more key components, all of said key components being utilized for defining said path through the levels of said tree-arranged storage array.

Claim 5

The system of claim 3 wherein said preprocessor means includes: a. means for providing An initial signal for the first processor in the cascade taking the place in structure of the probabilistic signal generated by each of the processors for the subsequent processor in the cascade, and b. means for encoding said initial signal into one or more key components.

Claim 6

The system of claim 4 including means for sequentially comprising the key components of the present input and probabilistic signals with the key components of input and probabilistic signals which have previously defined paths through the levels of said three-arranged storage array.

Claim 7

The system of claim 1 wherein said root level is directly addressable and said leaf level is addressable in a defined path extending from a storage unit of said root level.

Claim 8

The system of claim 6 including: preprocessor means for encoding said at least one input signal into one I key component, and each of said processors including; means for encoding said probabilistic signal into one X key component, said I key component providing means for directly addressing a storage unit of said root level and said X key component providing means for addressing a storage unit of said leaf level extending in a path from the addressed storage unit in said root level.

Claim 9

The system of claim 7 wherein said preprocessor means includes: a. means for generating an initial signal for the first processor in the cascade analogous to the probabilistic signal generated by each of the processors for the subsequent processor in the cascade, and b. means for encoding said initial signal into one X key component.

Claim 10

The system of claim 8 including means for sequentially comprising I and X key components of the present input and probabilistic signals with the I and X key components of input and probabilistic signals which have previously defined paths through the root and leaf levels of said tree-arranged storage array.

Claim 11

The system of claim 2 wherein the last processor in the cascade includes means for converting the probabilistic signal generated thereby to an actual output signal which is a best estimate of a desired response to an input signal applied to the first processor when the system is operated in an execution mode.

Claim 12

In a method of operating a trainable system of cascaded processors, the steps of: a. training a (k-1)th trainable non-linear processor to store therein (k-1)th statistical data based upon an applied first input signal and an applied desired response to such first input signal, b. executing said (k-1)th processor to generate from such stored (k-1)th statistical data a (k-1)th probabilistic signal which is a statistical estimate of the desired response to a second applied input signal, c. training a kth trainable non-linear processor to store therein kth statistical data based upon a third input signal comprising said first probabilistic signal generated by said (k-1)th processor and applied desired response to such third applied input signal applied thereto, d. executing said (k-1)th processor to generate from such stored (k-1)th statistical data a second probabilistic signal which is a statistical estimate of the desired response to a fourth applied input signal, and e. executing said kth processor to generate from such stored second statistical data an actual output signal which is a lower entropy estimate of the desired response to said fourth input signal when a fifth input signal comprising said second probabilistic signal generated by said (k-1)th processor is applied thereto.

Claim 13

In a method of operating a trainable system of cascaded processors, the steps of: a. training a (k-1)th trainable non-linear processor to store therein (k-1)th statistical data based upon an applied first input signal and an applied desired response to such first input signal, b. executing said (k-1)th processor to generate from such stored (k-1)th statistical data a (k-1)th probabilistic signal which is a statistical estimate of the desired response to a second applied input signal, c. training a kth trainable non-linear processor to store therein kth statistical data based upon a third input signal comprising said first probabilistic signal generated by said (k-1)th processor and applied desired response to such third applied input signal applied thereto, d. executing said (k-1)th processor to generate from such stored (k-1)th statistical data a second probabilistic signal which is a statistical estimate of the desired response to a fourth applied input signal, and e. executing said kth processor to generate from such stored second statistical data an actual output signal which is a lower entropy estimate of the desired response to said fourth input signal when a fifth input signal comprising said second probabilistic signal generated by said (k-1)th processor is applied thereto.

Claim 14

The process of claim 11 including the step of delaying at least one signal comprising said third input signal to provide at least one signal comprising said second input signal.

Claim 15

In a method of operating a trainable system of cascaded processors, the stEps of: a. storing first statistical data based upon an applied first input signal and an applied desired response to such first input signal in a first trainable non-linear processor operated in a training mode, b. generating from such stored first statistical data a first probabilistic signal which is a statistical estimate of the desired response to a second signal applied to said first processor operated in an execution mode, c. storing second statistical data based upon a third applied input signal comprising said first probabilistic signal generated by said first processor and applied desired response to such third input signal in a second trainable non-linear processor operated in a training mode, d. generating from such stored first statistical data a second probabilistic signal which is a statistical estimate of the desired response to a fourth signal applied to said first processor operated in an execution mode, and e. generating from such stored second statistical data an actual output signal which is a lower entropy estimate of the desired response to said fourth signal when a fifth input signal comprising said second probabilistic signal generated by said first processor is applied to said second processor operated in an execution mode.

Claim 16

In a method of operating a trainable system of cascaded processors, the stEps of: a. storing first statistical data based upon an applied first input signal and an applied desired response to such first input signal in a first trainable non-linear processor operated in a training mode, b. generating from such stored first statistical data a first probabilistic signal which is a statistical estimate of the desired response to a second signal applied to said first processor operated in an execution mode, c. storing second statistical data based upon a third applied input signal comprising said first probabilistic signal generated by said first processor and applied desired response to such third input signal in a second trainable non-linear processor operated in a training mode, d. generating from such stored first statistical data a second probabilistic signal which is a statistical estimate of the desired response to a fourth signal applied to said first processor operated in an execution mode, and e. generating from such stored second statistical data an actual output signal which is a lower entropy estimate of the desired response to said fourth signal when a fifth input signal comprising said second probabilistic signal generated by said first processor is applied to said second processor operated in an execution mode.

Claim 17

The method of claim 13 wherein the method of storing first statistical data includes the steps of: a. accumulating and storing the number of occurrences that each possible desired response has been associated with each same first input signal, and the method of storing second statistical data includes the steps of: b. accumulating and storing the number of occurrences that each possible desired response has been associated with each same third input signal.

Claim 18

In a method of operating a trainable system of cascaded processors, the steps of: a. encoding a first applied input signal into a plurality of key components, b. defining a path through the levels of a tree-arranged storage array of a first trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said first encoded input signal, c. storing first statistical data based upon an applied desired response to such first input signal in said storage unit at the leaf level of the tree-arranged storage array of said first processor, d. encoding a second applied input signal into a plurality of key components, e. defining a path through the levels of the tree-arranged storage array of said first processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said second encoded input signal, f. generating from such stored first statistical data a first probabilistic signal which is a statistical estimate of the desired response to said second input signal, g. encoding a third applied input signal comprising said first probabilistic signal generated by said first processor into a plurality of key components, h. defining a path through the levels of a tree-arranged storage array of a second trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said third encoded input signal, i. storing second statistical data based upon an applied desired response to such third input signal in said storage unit at the leaf level of the tree-arranged storage array of said second processor, j. encoding a fourth applied input signal into a plurality of key components, k. defining a path through the levels of the tree-arranged storage array of said first processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said fourth encoded input signal, l. generating from such stored first statistical data a secOnd probabilistic signal which is a statistical estimate of the desired response to said fourth input signal, m. encoding a fifth applied input signal comprising said second probabilistic signal generated by said first processor into a plurality of key components, n. defining a path through the levels of the tree-arranged storage array of said second processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said fifth encoded input signal, and o. generating from such stored second statistical data an actual output signal which is a lower entropy estimate of the desired response to said fourth input signal.

Claim 19

In a method of operating a trainable system of cascaded processors, the steps of: a. encoding a first applied input signal into a plurality of key components, b. defining a path through the levels of a tree-arranged storage array of a first trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said first encoded input signal, c. storing first statistical data based upon an applied desired response to such first input signal in said storage unit at the leaf level of the tree-arranged storage array of said first processor, d. encoding a second applied input signal into a plurality of key components, e. defining a path through the levels of the tree-arranged storage array of said first processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said second encoded input signal, f. generating from such stored first statistical data a first probabilistic signal which is a statistical estimate of the desired response to said second input signal, g. encoding a third applied input signal comprising said first probabilistic signal generated by said first processor into a plurality of key components, h. defining a path through the levels of a tree-arranged storage array of a second trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said third encoded input signal, i. storing second statistical data based upon an applied desired response to such third input signal in said storage unit at the leaf level of the tree-arranged storage array of said second processor, j. encoding a fourth applied input signal into a plurality of key components, k. defining a path through the levels of the tree-arranged storage array of said first processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said fourth encoded input signal, l. generating from such stored first statistical data a secOnd probabilistic signal which is a statistical estimate of the desired response to said fourth input signal, m. encoding a fifth applied input signal comprising said second probabilistic signal generated by said first processor into a plurality of key components, n. defining a path through the levels of the tree-arranged storage array of said second processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said fifth encoded input signal, and o. generating from such stored second statistical data an actual output signal which is a lower entropy estimate of the desired response to said fourth input signal.

Claim 20

The method of claim 15 wherein the method of defining a path through the levels of the tree-arranged storage arrays of said first and second processors includes the steps of: a. directly addressing a storage unit in a first root level of the storage array, and b. addressing the storage unit in said leaf level in a defined path extending from the storage unit of said root level.

Claim 21

The method of claim 15 wherein the method of storing first statistical data includes the steps of: a. accumulating and storing the number of occurrences that each possible desired response has been associated with each same first input signal, and the method of storing second statistical data includes the steps of: b. accumulating and storing the number of occurrences that each possible desired response has been associated with each same third input signal.

Claim 22

In a method of operating a trainable system of cascaded processors, the steps of: a. encoding a first applied input signal into a plurality of key components, b. defining a path through the levels of a tree-arranged storage array of a first trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said first encoded input signal, c. sequentially comparing the key components of said first input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said first processor. d. storing statistical data based upon an applied desired response to such first input signal in said storage unit at the leaf level of the tree-arranged storage array of said first processor when no such path has been previously defined. e. updating the statistical data stored in said storage unit at the leaf level of the tree-arranged storage array of said first processor when such a path has been previously defined. f. encoding a second applied input signal into a plurality of key components, g. defining a path through the levels of the tree-arranged storage of said first processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said second encoded input signal, h. sequentially comparing the key components of said second input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said first processor to locate first statistical data which provides a best statistical estimate for said first processor of the desired response to said second input signal. i. generating from such located first statistical data a first probabilistic signal, j. encoding a third applied input signal comprising said first probabilistic signal generated by said first processor into a plurality of key components, k. defining a path through the levels of a tree-arranged storage array of a second trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said third encoded input signal, l. sequentially comparing the keY components of said third input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said second processor. m. storing statistical data based upon an applied desired response to such third input signal in said storage unit at the leaf level of the tree-arranged storage array of said second processor when no such path has been previously defined. n. updating the statistical data stored in said storage unit at the leaf level of the tree-arranged storage array of said second processor when such a path has been previously defined. o. encoding a fourth applied input signal into a plurality of key components, p. defining a path through the levels of the tree-arranged storage array of said first processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said fourth encoded input signal, q. sequentially comparing the key components of said fourth input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said first processor to locate first statistical data which provides a best statistical estimate for said first processor of the desired response to said fourth input signal. r. generating from such located first statistical data a second probabilistic signal, s. encoding a fifth applied input signal comprising said second probabilistic signal generated by said first processor into a plurality of key components, t. defining a path through the levels of the tree-arranged storage array of said second processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said fifth encoded input signal, u. sequentially comparing the key components of said fifth input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said second processor to locate second statistical data which provides a lower entropy estimate of the desired response to said fourth input signal than provided by said first processor. v. generating from such located second statistical data an actual output signal.

Claim 23

In a method of operating a trainable system of cascaded processors, the steps of: a. encoding a first applied input signal into a plurality of key components, b. defining a path through the levels of a tree-arranged storage array of a first trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said first encoded input signal, c. sequentially comparing the key components of said first input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said first processor. d. storing statistical data based upon an applied desired response to such first input signal in said storage unit at the leaf level of the tree-arranged storage array of said first processor when no such path has been previously defined. e. updating the statistical data stored in said storage unit at the leaf level of the tree-arranged storage array of said first processor when such a path has been previously defined. f. encoding a second applied input signal into a plurality of key components, g. defining a path through the levels of the tree-arranged storage of said first processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said second encoded input signal, h. sequentially comparing the key components of said second input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said first processor to locate first statistical data which provides a best statistical estimate for said first processor of the desired response to said second input signal. i. generating from such located first statistical data a first probabilistic signal, j. encoding a third applied input signal comprising said first probabilistic signal generated by said first processor into a plurality of key components, k. defining a path through the levels of a tree-arranged storage array of a second trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said third encoded input signal, l. sequentially comparing the keY components of said third input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said second processor. m. storing statistical data based upon an applied desired response to such third input signal in said storage unit at the leaf level of the tree-arranged storage array of said second processor when no such path has been previously defined. n. updating the statistical data stored in said storage unit at the leaf level of the tree-arranged storage array of said second processor when such a path has been previously defined. o. encoding a fourth applied input signal into a plurality of key components, p. defining a path through the levels of the tree-arranged storage array of said first processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said fourth encoded input signal, q. sequentially comparing the key components of said fourth input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said first processor to locate first statistical data which provides a best statistical estimate for said first processor of the desired response to said fourth input signal. r. generating from such located first statistical data a second probabilistic signal, s. encoding a fifth applied input signal comprising said second probabilistic signal generated by said first processor into a plurality of key components, t. defining a path through the levels of the tree-arranged storage array of said second processor to a storage unit in the leaf level thereof in accordance with at least two of said key components of said fifth encoded input signal, u. sequentially comparing the key components of said fifth input signal with the key components of input signals which have previously defined paths through the levels of the tree-arranged storage array of said second processor to locate second statistical data which provides a lower entropy estimate of the desired response to said fourth input signal than provided by said first processor. v. generating from such located second statistical data an actual output signal.

Claim 24

The method of claim 18 wherein the method of storing first statistical data includes the steps of: a. accumulating and storing the number of occurrences that each possible desired response has been associated with each same first input signal, and the method of storing second statistical data includes the steps of: b. accumulating and storing the number of occurrences that each possible desired response has been associated with each same third input signal.

Claim 25

In a method of operating a system of cascaded processors, the steps of: a. encoding a first applied input signal into a plurality of key components, b. defining a path through the levels of a tree-arranged storage array of a first non-linear signal processor said storage array having a plurality of levels and said path defined to locate a storage unit in the leaf level thereof in accordance with at least two of said key components of said first encoded input signal, c. generating from first statistical data stored in such storage unit of the first processor a probabilistic signal which is a statistical estimate of the desired response to said first input signal, d. encoding a second applied input signal comprising said probabilistic signal generated by said first processor into a plurality of key components, e. defining a path through the levels of a tree-arranged storage array of a second trainable non-linear signal processor said storage array having a plurality of levels and said path defined to locate a storage unit in the leaf level thereof in accordance with at least two of said key components of said third encoded input signal, f. generating from such stored second statistical data stored in such storage unit of the second processor an actual output signal which is a lower entropy estimate of the desired response to said first input signal.

Claim 26

In a method of operating a system of cascaded processors, the steps of: a. encoding a first applied input signal into a plurality of key components, b. defining a path through the levels of a tree-arranged storage array of a first non-linear signal processor said storage array having a plurality of levels and said path defined to locate a storage unit in the leaf level thereof in accordance with at least two of said key components of said first encoded input signal, c. generating from first statistical data stored in such storage unit of the first processor a probabilistic signal which is a statistical estimate of the desired response to said first input signal, d. encoding a second applied input signal comprising said probabilistic signal generated by said first processor into a plurality of key components, e. defining a path through the levels of a tree-arranged storage array of a second trainable non-linear signal processor said storage array having a plurality of levels and said path defined to locate a storage unit in the leaf level thereof in accordance with at least two of said key components of said third encoded input signal, f. generating from such stored second statistical data stored in such storage unit of the second processor an actual output signal which is a lower entropy estimate of the desired response to said first input signal.

Claim 27

The method of claim 15 wherein the method of defining a path through the levels of the tree-arranged storage arrays of said first and second processors includes the steps of: a. directly addressing a storage unit in a first root level of the storage array, and b. addressing the storage unit in said leaf level in a defined path extending for the storage unit of said root level.

Claim 28

In a method of operating a system of cascaded processors, the steps of: a. encoding a first applied input signal into a plurality of key components, b. defining a path through the levels of a tree-arranged storage array of a first trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said first encoded input signal, c. sequentially comparing the key components of said first input signal with the key components which define paths through the levels of the tree-arranged storage array of said first processor to locate first statistical data which provides a best statistical estimate for said first processor of the desired response to said first input signal. d. generating from such located first statistical data a probabilistic signal, e. encoding a second applied input signal comprising said probabilistic signal generated by said first processor into a plurality of key components, f. defining a path through the levels of a tree-arranged storage array of a second trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said encoded input signal, g. sequentially comparing the key components of said second input signal with the key components which define paths through the levels of the tree-arranged storage array of said second processor to locate second statistical data stored in such storage unit of the second processor which provides a lower entropy estimate of the desired response to said first input signal than provided by said first processor. h. generating from such located second statistical data an actual output signal.

Claim 29

In a method of operating a system of cascaded processors, the steps of: a. encoding a first applied input signal into a plurality of key components, b. defining a path through the levels of a tree-arranged storage array of a first trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said first encoded input signal, c. sequentially comparing the key components of said first input signal with the key components which define paths through the levels of the tree-arranged storage array of said first processor to locate first statistical data which provides a best statistical estimate for said first processor of the desired response to said first input signal. d. generating from such located first statistical data a probabilistic signal, e. encoding a second applied input signal comprising said probabilistic signal generated by said first processor into a plurality of key components, f. defining a path through the levels of a tree-arranged storage array of a second trainable non-linear signal processor said storage array having a plurality of levels and said path defined to a storage unit in the leaf level thereof in accordance with at least two of said key components of said encoded input signal, g. sequentially comparing the key components of said second input signal with the key components which define paths through the levels of the tree-arranged storage array of said second processor to locate second statistical data stored in such storage unit of the second processor which provides a lower entropy estimate of the desired response to said first input signal than provided by said first processor. h. generating from such located second statistical data an actual output signal.

Claim 30

A trainable system of cascaded processors comprised of: a plurality of non-linear signal processors in cascade, each processor including: a. means for applying at least one input signal from an external source and at least one probabilistic signal generated by the previous processor in the cascade thereto; b. a multi-level tree-arranged storage array having at least a root level and a leaf level; c. means for defining a path through the levels of the tree in accordance with said input and probabilistic signals; and d. means for generating at least one output signal comprising a probabilistic signal for the subsequent processor in the cascade.

Claim 31

A trainable system of cascaded processors comprised of: a plurality of non-linear signal processors in cascade, each processor including: a. means for applying at least one input signal from an external source and at least one probabilistic signal generated by the previous processor in the cascade thereto; b. a multi-level tree-arranged storage array having at least a root level and a leaf level; c. means for defining a path through the levels of the tree in accordance with said input and probabilistic signals; and d. means for generating at least one output signal comprising a probabilistic signal for the subsequent processor in the cascade.

Claim 32

The system of claim 23 wherein each processor is operable in a training mode additionally including means for applying at least one desired output signal from an external source thereto when such processor is operated in said training mode.

Claim 33

The system of claim 24 wherein said leaf level includes means for accumulating and storing the number of occurrences that each desired output signal was associated with each of said defined paths during training.

Claim 34

The system of claim 25 wherein said means for generating at least one output signal is responsive during execution to said means for storing the number of occurrences that each desired output signal was associated with each of said defined paths during training.