Computer controlled imaging system

Claim 2

A computer controlled imaging system responsive to the position and orientation of a scene recognition means characterized by: a data base (200) having a defined area (208) with a digital three coordinate system and a library (212) having a plurality of two dimensional image frames of electromagnetic spectrum band representations of images; said defined area part of said data base having scene composition data which defines location and size of each of said image frames relative to said coordinate system; simulation means (202) for supplying observation data defining the location and the field of view of said scene recognition means relative to said defined area coordinate system; interface means (204) for receiving said observation data; field of view processor means (206) associated with said interface means (204) and said data base (208), said field of view processor means including means for determining which of said image frames are included in said field of view and their respective ranges relative to said observation data and said frame data to compute corner coordinates which correspond to the defined area coordinates of the corners of each said included image frames; controller means (210) connected to said field of view processor (206) and including means to receive data therefrom said data including the identity of said image frames included in said field of view, said ranges of said included image frames and said coordinates of each of said included image frames; said controller means being further connected to said library part (212) of said data base; processing channel means (214 to 218) controlled by said controller means connected to said library part (212) of said data base, and wherein said library part (212) of said data base is connected between said controller (210) and said processing channel means (214 to 218), said channel means operating to map said included frames to the enclosed space defined by the corresponding ones of said corner coordinates; and scene construction means (220) connected to said processing channels (214 to 218) for assembling an output scene based on said ranges of said included frames on a digital basis such that the included frame nearest said scene recognition means includes more distant ones of said included frames.

Claim 3

System according to claim 1, character- ized in that said image frames are image representations selected from a group of electromagnetic wavelength bands consisting of IR, visual, millimeter wave or SAR domain.

Claim 4

System according to claim 1, character- ized by a human-recognizable display device for displaying said scene.

Claim 5

System according to claim 3, character- ized in that said display device is a video output device.

Claim 6

System according to any of claims 1 to 4, charac-terized in that said output scenes are produced in a continual sequential stream and correspond to any movement of said scene recognition means in said defined area.

Claim 7

System according to claim 5, characterized in that said sequential stream of output scenes are produced at a rate sufficient to appear as a continuous display to a human observer.

Claim 8

System according to claim 5, characterized in that said output scenes are produced in real-time response to the position and orientation, including changes therein, of said scene recognition means in said defined area.

Claim 9

System according to claim 5, characterized in that said movement of said scene recognition means in said defined area represents the simulated movement of a vehicle and wherein said output scenes comprise a continuous video display of the field of view of the vehicle operator.

Claim 10

System according to claim 1, characterized in that at least one of said image frames is a pictorial representation of a two dimensional object photographed from a predetermined average aspect angle.

Claim 11

System according to claim 9, characterized in that said aspect angle is about minus 15 degrees.

Claim 12

System according to claim 1, characterized in that some of said image frames are a series of pictorial representations of a particular object photographed at different height intervals relative to the vertical axis thereof.

Claim 13

System according to claim 1, characterized in that at least one of said image frames is a pictorial representation of a group of objects to thereby increase detail simulation thereof.

Claim 14

System according to claim 1, characterized in that some of said frames are pictorial representations of terrain surfaces.

Claim 15

System according to claim 13, character- ized in that said some of said image frames for a particular terrain surface are in a group in which variations of said particular terrain surface from frame to frame provide motion simulation effects including blowing wind and running water.

Claim 16

System according to claim 13, character- ized in that at least some of said frames are pictorial representations of textured surfaces such as water and grass.

Claim 17

System according to claim 15, character- ized in that said defined area part of said data base includes defined elongated surface areas for natural and man-made terrain features, said frames of pictorial representations of textured surfaces being appropriately provided to represent said terrain features.

Claim 18

System according to claim 1, characterized by portraying a large three dimensional, two axis object, by storing a series of pictures in increments of small angles in both azimuth and elevation.

Claim 19

System according to claim 17, character- ized in that said small angles are on the order of one degree.

Claim 20

System according to claim 17, character- ized in that said small angles are precisely obtained by taking pictures in connection with rotating a model of said large object on rotatable tables having vertical and horizontal axes.

Claim 21

System according to claim 17, character- ized in that said large object is broken down into separate 2D subsurfaces which are assembled by at least two of said one or more processing channels.

Claim 22

System according to claim 17, character- ized in that a series of photographs are taken of said large objects at angular azimuth intervals at a fixed elevation.

Claim 23

System according to claim 12, character- ized in that said angular azimuth intervals are on the order of 30 degrees.

Claim 24

System according to claim 1, character- ized in that at least one of said image frames contains one or more light sources.

Claim 25

System according to claim 1, character- ized in that at least one of said frames has a translucent characteristic for adding a special effect to a scene.

Claim 26

System according to claim 24, character- ized in that said special effects comprise fog, dust, smoke shadow or haze.

Claim 27

System according to either of claims 24 or 25, characterized in that said translucent characteristic is stored as a mask which defines the outline, shape and transmissivity factor thereof.

Claim 28

System according to claim 26, charac- terized in that a series of said frames have varying mask shapes to generate a motion effect.

Claim 29

System according to claim 1, further charac- terized by means for producing occlusion of the contents of distant frames by those in closer frames in said scene.

Claim 30

System according to claim 28, character- ized in that at least one of said frames portrays an elongated object, said defined area part of said data base having multiple range points for different sections of said elongated object with the particular multiple range point closest to said observer having priority relative to occlusion comparison with other ones of said included frames.

Claim 31

System according to claim 29, character- ized in that two of said multiple range points are provided for opposite ends of said elongated object.

Claim 32

System according to claim 1, character- ized in that said image frames of electromagnetic spectrum band representations are in analog form, one of said at least one processing channels further comprises: A/D converter means for converting said representations to digital data form; buffer means accepting said digital data in either the X or Y axis; and means for mapping the image of said digital data to the corner coordinates of a video screen.

Claim 33

System according to claim 31, character- ized by means for modifying the intensity values of said data in said buffer means.

Claim 34

System according to claim 1, character- ized in that said image frames of electromagnetic spectrum band representations are in digital data form; and wherein one of said at least one processing channel includes buffer means for accepting said digital data in either the X or Y axis, and means for mapping the image of said digital data to the corner coordinates of said display means.

Claim 35

System according to claim 1, character-ized in that said scene construction means comprises channel combiner means for multiplexing signals from a plurality of processing channels and corresponding range information from said controller means, said channel combiner further comprising switch means for selecting data from said plurality of processing channels outputting scene data relative to range in the field of view of said scene recognition means.

Claim 36

System according to claim 1, character- ized by means for warping said images in said image frames requisite to said scene representation.

Claim 37

System according to claim 35, character- ized by means for accomplishing both linear and perspective image warping as desired.

Claim 38

System according to claim 1 or one of the following claims, characterized by means for mapping an image to a viewing plane in a coordinate system having an observation point on a main axis thereof and a viewing plane extending normal to said axis and spaced from said viewing point, wherein a rectangular digital input image arranged in columns and rows and represented as a matrix of pixel intensity values is used, and wherein said input image is on the side of said viewing plane opposite from said observation point and corner coordinates related to said main axis, with said means comprising: means for making two orthogonal passes with said first pass is over said input image to form an intermediate image and wherein said second pass is over said intermediate image to form an output image; means for making line transformations in said first pass wherein said input image columns are converted to intermediate image columns having lengths derived from said input image columns such that there are respective ratios defining the lengths of said input image columns relative to said intermediate image columns; means for making line transformations in said second pass wherein the intermediate image rows are converted to output image rows on said viewing screen having lengths derived from said input image rows such that there are respective ratios defining the lengths of said intermediate image rows relative to said output image rows; means for calculating for each of said line transformations on the basis of said ratios the number of input pixels required respectively from said columns of said input image and said rows of said intermediate image to create output pixels respectively for said intermediate image columns and said final image rows; means for mapping said input pixels sequentially and fractionally to form said output pixels for each cf said line transformations; and means for summing and averaging the intensity values of said input pixels applicable to each output pixel and respectively accruing the composite values so obtained to said output pixels.

Claim 39

System according to one of the claims 1 to 36, charac-terized by means for mapping a digital input orthogonal image arranged in columns and rows represented as a rectangular pixel matrix of intensity values in a plane to any convex quadrilateral, with said means comprising: means for performing sequentially two orthogonal scanning passes of said image wherein said first pass forms an intermediate image from said input image and wherein said second pass forms an output image from said intermediate image; wherein said first pass further accomplishes transformations wherein the columns of said input image are converted to aligned intermediate vertical columns having lengths and positions determined from the input image coordinates and the coordinates of said quadrilateral; wherein said second pass further processes said intermediate image and makes line transformations whereby said rows are converted to aligned horizontal output rows having lengths and positions determined from the corners of said intermediate image and said coordinates of said quadrilateral; means for adjusting the number of input pixels required to create an output pixel; and means for mapping said input pixels sequentially and fractionally to said output pixels for each of said line transformations.

Claim 40

System according to one of the claims 1 to 36, characterized by means for mapping a digital input image to a viewing plane wherein said image comprises elements arranged in columns and rows representing a matrix of pixel intensity values in a 3D coordinate system having an observation point on a main axis thereof and a viewing plane extending normal to said axis and spaced from said observation point, said system defining a reference plane intersecting said main axis and extending normal to said viewing plane, with said means comprising: means for sequentially and orthogonally projecting said columns of said input image to said reference plane to form first-pass object lines in said reference plane; means for processing each of said first pass object lines in a manner which simulates projecting divergent lines having a separation of at least one pixel from said origin through said viewing plane and said corresponding first-object line such that the intersections of said divergent lines with said viewing plane describe an intermediate image line having segments between said divergent lines having a length equal to the number of pixels separating said divergent lines; means for obtaining the pixel ratio for each of said first pass object line segments relative to the corresponding one of said intermediate image line segments; means for mapping the pixels of each column sequentially and fractionally to the first pass object line segments of the corresponding first pass object line; means for projecting divergent planes from said origin through rows of said viewing plane, said divergent planes having a separation of at least one pixel, each of said divergent planes intersecting said viewing plane to form a final image line and intersecting said input image to form a second pass object line; means for processing each of said second pass object lines by extending second pass divergent lines from said origin through columns of said viewing plane having a separation of at least one pixel and through said second pass object line, said second pass divergent lines intersecting said viewing plane to form a horizontally extending correctly positioned final image line with segments each having a length equal to the number of pixels separating said second pass divergent lines and intersecting said second pass object line to form segments; means for obtaining the pixel ratio for each of said second pass object line segments relative to the corresponding one of said final image line segments, for each of said intermediate image rows; and means for mapping the pixels of the row sequentially and fractionally to the second pass object line segments of the corresponding second pass object line.

Claim 41

System according to one of the claims 1 to 36, characterized by means for mapping in a 3D coordinate system having a viewing plane extending normal to said axis and spaced from said viewing point, said system having a reference plane containing said main axis and extending normal to said viewing plane, said method being applied to a digital input image arranged in columns and rows and represented as a matrix of pixel intensity values, said input image being on the opposite side of said viewing plane from said viewing point, said means being applicable to map said input image to said viewing plane and comprising: means for successively and orthogonally projecting said columns of said input image to said reference plane to form first pass object lines in said reference plane, means for processing each of said first pass object lines in sequence by extending radiating rays in said reference plane from said origin through said viewing plane and said first pass object line, said radiating rays intersecting said viewing plane to form an intermediate image line with segments each having the length of one pixel and intersecting said first pass object line to form segments each having a different length, means for prorating the total number of said input image rows with respect to said first pass object line segments to obtain the pixel ratio for each of said first pass object line segments relative to the corresponding one of said intermediate image line segments, for each of said input columns mapping the pixels thereof sequentially and fractionally to the first pass object line segments of the corresponding first pass object line, means for summing and averaging for each segment of each of said first pass object lines the intensity values of each pixel mapped thereto and applying the composite value to the corresponding intermediate image line segment, means for processing said input image by extending radiating planes from said origin through rows of said viewing plane having one pixel separation and through said input image, each of said radiating planes intersecting said viewing plane to form a final image line and intersecting said input image to form a second pass object line, means for processing each of said second pass object lines by extending second pass radiating rays from said origin through columns of said viewing plane having one pixel separation and through said second pass object line, said second pass radiating rays intersecting said viewing plane to form a horizontally extending correctly positioned final image line with segments each having the length of one pixel and intersecting said second pass object line to form segments each having a different length, means for prorating the total number of said intermediate image rows with respect to said second pass object line segments to obtain the pixel ratio for each of said second pass object line segments relative to the corresponding one of said final image line segments, for each of said intermediate image rows mapping the pixels of the row sequentially and fractionally to the second pass object line segments of the corresponding second pass object line, means for summing and averaging for each segment of each of said second pass object lines the intensity values of each pixel mapped thereto applicable to each final image pixel and applying the composite value to the corresponding final image line segment.

Claim 42

System according to claim 37, character- ized in that said mapping has a preorientation of 0, 90, 180 or 270 degrees depending on which said orientation yields the largest area for the four possible corresponding intermediate images.

Claim 43

System according to one of the claims 1 to 36, characterized by means for mapping a digital input image represented as a rectangular grid of intensity values in a plane to any convex of quadrilateral, with said means comprising: means for projecting each output pixel in said quadrilateral into said grid, and means for computing the intensity of each said output pixel from the region on said grid occupied by said output pixel, said intensity of each said output pixel being an area weighted average of the intensity values of all of the whole and fractional regions of said grid upon which each said output pixel is projected.

Claim 44

System according to claim 42, character- ized in that the intensities for fractional regions are bilinear interpolations.

Claim 45

System according to claim 1 or one of the following claims, characterized by means for image address to memory address mapping of a two axis square array image with said means mapping pixels of said image to 2 N independently accessible memory banks such that memory elements corresponding to any 2 N consecutive pixels of any row and any column of said image reside in different banks and can be accessed simultaneously.

Claim 46

System according to claim 44, character- ized in that said pixels are spirally mapped to said banks.

Claim 47

System according to claim 44, character- ized in that said mapping involves scanning said array from either the horizontal direction or the vertical direction.

Claim 48

System according to one of claims 1 to 43, characterized by means for image address to memory address mapping with said means de- multiplexingly mapping pixels of a two axis (A x A) square array image into 2 N memory banks such that memory elements corresponding to any 2 N consecutive pixels of any row and any column of said image reside in different banks and can be accessed simultaneously.

Claim 49

System according to claim 47, character- ized by multiplexingly scanning said banks to sequentially output sets of memory elements corresponding to sets of 2 N consecutive pixels of said image rows, said memory elements in each said set being output simultaneously.

Claim 50

System according to claim 47, character- ized in that said scanning yields a stream of corresponding row or column oriented pixels with each pixel being identified by its image row and column positions, assigning a designated memory bank and an address thereof to each of said pixels by demultiplexing said stream which involves processing each said pixel to obtain (1) said designated memory bank by adding modulo N the lower N bits of said image row and column positions and (2) said address of said designated memory bank by summing said image row with a number derived by ANDING said image column with a number equal to A/2 .

Claim 51

System according to claim 48, character- ized in that said scanning yields a stream of memory elements with each of said memory elements being identified by the particular bank in which it resides and its address therein, assigning corresponding image row and column positions to each of said memory elements by multiplexing said sets which involves processing each said memory element to obtain the corresponding image column by a rotation function determined from the relationship between said particular bank and the lower N bits of said particular address, said image column then being the sum of said rotation function and a number derived by ANDING said particular address with (A 2 /A N - A) and dividing the resultant by A/2 N , and each said memory element being processed to obtain the corresponding image row by ANDING said bank address with a number equal to A.