US 2017/0163733 A1 - System And Method For Data Management Structure Using Auditable Delta Records In A Distributed Environment - The Lens

System And Method For Data Management Structure Using Auditable Delta Records In A Distributed Environment

Published: Jun 8, 2017
Earliest Priority: Dec 02 2015
Family: 2
Cited Works: 0
Cited by: 50
Cites: 0
Additional Info: Full text

Abstract

A system and method of recording data from a number of devices in a distributed network system in a manner adaptable for auditing the device output. The devices may include one or more control, sensor, edge, or peripheral computing devices physically separate in the distributed network system and in communication with a control server. Such distributed networks systems are common in SCADA or IoT applications. The content stream of data records output from the devices are recorded; a payload stripped stream of data records which are stripped of the payload are recorded and preferably retained by an escrow service. The metadata of the data records includes the hash value of one or more predecessor data records. The hash values are calculated based on the payload and a linkage function, preferably a cryptographic function. A comparison of hash values of the payload stripped stream and the content stream provides the audit ability.

Claims

A method of recording data from a number of devices in a distributed network system in a manner adaptable for audit, comprising:
recording a content stream of data records output from said devices where each record has a payload segment including content from said devices and a metadata segment,
where the metadata segment includes the hash value of a predecessor record in the stream;

recording a payload stripped stream of said data records where the payload stripped stream does not include the payload segment; and

storing said a payload stripped stream as an escrow file.
The method of claim 1, wherein a hash value in a record is calculated based at least on a payload in a predecessor record and a linkage function.
The method of claim 2, wherein the linkage function includes a cryptographic function applied to the payload of one or more predecessor records.
The method of claim 2, wherein the linkage function changes periodically.
The method of claim 1, wherein the content stream and payload stripped stream are recorded in parallel to separate databases.
The method of claim 1, wherein at least some of the data records are encrypted prior to recording.
The method of claim 1, wherein at least some of the data records are stored in a buffer and later stored in a database
The method of claim 1, wherein the escrow file is stored with a third party escrow service.
The method of claim 1, wherein the devices include one or more control, sensor, edge, or peripheral computing devices physically separate in the distributed network system and in communication with a control server.
The method of claim 1, including auditing the content stream of data records by determining the hash values of each content data record, determining the hash values of the payload stripped stream and comparing the content stream hash values with the payload stripped stream hash values.
The method of claim 1, including recording one or more subchains of the content stream of data records using a fork operation.
The method of claim 1, comparing two subchains to establish a time wise correlation of events.
The method of claim 1, wherein the first record in said content stream of data records output from each of said devices is a genesis block created by a device and includes a strong random number.
The method of claim 1, wherein the first record in said content stream of data records output from each of said devices is a genesis block transferred to a device in a trusted manner and is communicated to an escrow service in a trusted manner.
A system collecting audit records from a number of devices comprising:
a number of devices in a distributed network system which communicate with a data repository and generate a content stream of data records, each data record in the content stream having a payload segment and a metadata segment,
the devices and central repository operable to generate a payload stripped stream of data records stripped of the payload segment, where the metadata segment includes the hash value of a predecessor data record;

a communication network operable for communication between said devices and said central repository;

a content database operable coupled to said central repository for recording said content stream from said devices; and

an audit database operable for recording said payload stripped stream.
The system of claim 15, wherein the hash value in a record is calculated based at least on a payload in a predecessor record and a linkage function.
The system of claim 16, wherein the linkage function includes a cryptographic function.
The system of claim 15, wherein the audit database is stored with a third party escrow agent physically remote from said distributed network system.
The system of claim 15, wherein the devices include one or more control, sensor, edge, or peripheral computing devices physically separate in the distributed network system and in communication with data repository.
A method of auditing a stream of data records generated by a number of devices in a distributed network system comprising:
recording a content stream of data records output from said devices where each record has a payload segment including content from said devices and a metadata segment, where the metadata segment includes the hash value of a predecessor record in the stream;

recording a payload stripped stream of data records where each record has a metadata segment, where the metadata segment includes the hash value of a predecessor record in the stream;

determining the hash values of content stream data records, determining the hash values of the payload stripped stream data records and comparing the content stream hash values with the payload stripped stream hash values.
The method of claim 20, wherein the content stream hash values are compared with the respective payload stripped stream hash values.
The method of claim 20, including establishing multiple subchains of the payload stripped stream using fork and join operands and determining a time wise correlation between two or more subchains.
The method of claim 20, wherein at least some of the data records are encrypted prior to recording.
The method of claim 20, wherein the hash value of a record is determined based at least on a payload in a predecessor record and a linkage function.
The method of claim 20, wherein the linkage function is a cryptographic function.
The method of claim 20, wherein the linkage function changes periodically.
The method of claim 26, wherein the periodic change in linkage function is based on a protocol known to the distributed network system and an escrow service.