GB2435945A

GB2435945A - Transparent encryption and zipping file management system

Info

Publication number: GB2435945A
Application number: GB0604389A
Authority: GB
Inventors: Ahmed Eltigani
Original assignee: Individual
Current assignee: Individual
Priority date: 2006-03-04
Filing date: 2006-03-04
Publication date: 2007-09-12
Anticipated expiration: 2026-03-04
Also published as: GB2435945B; GB0604389D0

Abstract

The Encrypting and Zipping File System (EzFs) EzFs is a user-level pseudo file system that maps and exposes a local or remote physical folder as a virtual namespace via Universal naming Convention (UNC) or mapped drive letter to provide applications transparent file-level encryption and compression. An application I/O requests to files in the virtual namespace are tunneled to physical files in the underlying mapped physical folder. The physical files store application data in encrypted, compressed or clear format depending on the application request. EzFs manages the virtual name space as a transparent cache of the target folder and extends the functionality of the underlying target folder beyond the target volume capabilities. EzFs is able to provide transparent encryption of files via Windows NTFS Encrypting File System (EFS) functionality. Encryption is provided by bridging a local NTFS folder supporting Windows Encrypting File System (EFS) and a local or remote physical folder on a different media or computer. EzFs temporarily caches files in the local NTFS volume as encrypted files, extracts and writes the encrypted data raw blocks into the target files. The entire process of importing and exporting encrypted files from and to the local NTFS folder is totally transparent to applications. The EzFs file system bridging and data transformation technology enables it to extend Windows NTFS EFS feature to other file systems and storage platforms that don't necessarily support it e.g. FAT. This mechanism allows EzFs to provide encryption functionality independent of the volume or file system capabilities that hosts the actual file and enables mobility and transportability of Windows NTFS EFS encrypted files across machines and devices.

Description

Summary of the invention

EzFs exposes a virtual namespace via Universal Naming Convention (UNC) for a physical folder to provide transparent encryption and compression to applications. By using UNC naming convention the EzFs virtual folder can be accessed as a network resource and allows EzFs to full integration with the Windows infrastructure and Offline Files, it also allows user to access the EzFs virtual folder via a single name across multiple computers. This feature is important for mobile data access, such as Flash Drives, to provide users with a single consistent name across multiple computers. The virtual namespace directory hierarchy is mapped and stored directly in the physical folder. File attributes and sizes are preserved Ofl directory listings and attribute lookup requests.

One of the salient features of EzFs is to provide transparent file-level encryption and compression of files as data is read and written via its namespace. An application accessing a file in the EzFs narnespace continues to access the clear text unaware of the data transformation that alters the physical representation of the file. EzFs leverages existing encryption and security technologies and searniessly integrates with Windows Encrypting File System (EFS) to provide security on storage platforms that lack support for these technologies. For example, Flash Drives are typically formatted and accessed as FAT file system which lacks support for encryption and compression. EzFs secures data on the Flash Drive via Windows NTFS EFS functionality as described later.

Figure 1 depicts EZFS overall architecture. Applications access a normal drive letter or UNC path that resembles a real physical volume. EZFS manages the underlying physical folder and exposes the same namespace except files are read and written in clear text, compressed or encrypted format. Because EZFS only manipulates the data streams the underlying physical namespace hierarchy remains intact. Files are stored in the physical target storage volume in Windows EFS backup format. When applications access files through EZFS they receive the data in clear text format without the knowledge on how the files are actually stored on the target volume. EZFS transparently manages the import and export processes of moving the raw encrypted data blocks from and to the target volume into cached files on a local NTFS volume as EFS files to decipher the actual text that is then returned to the application. Application write calls are first applied to the cached copy in the local NTFS volume and once the application closes it write handle to the file EZFS exports the raw encrypted data blocks to the target volume and stores the file data in Windows EFS backup format. EZFS can also propagate each write operation change by directly reading the raw encrypted data blocks from the local NTFS volume and patching the target file on the target folder via differencing technique after every write operation. However, it's more efficient to wait until the file handle that issued the write operations is closed by the application in order to batch multiple write operations into a single update of the target physical file. Note, EzFs doesn't wait until the entire file is closed but rather when file handle instance that issued a write operation is closed. This allows EZFS to update files instantly that remain open by the application for a long period of time. The entire data transformation process is totally transparent to the application and file attributes, including size and timestamps, are preserved across lookup and query operations to preserve the Windows file system semantics.

There are many configurations that can benefit from EzFs data transformation and transparent file system bridging functionality. Securing Flash drives or removal media is a key scenario that can benefit from such technology. NTFS is an unsuitable file system for removable media and flash drives for several reasons. However, NTFS provides excellent data encryption and transparency to users that make EFS simple to use. EzFs leverages the benefits of NTFS encryption and transparency and extends it to removable media using the FAT file system format and other file systems that lack such functionality.

EzFs provides protection to files stored on network file servers and client-side Offline Files. Figure 2 illustrates how files are transmitted over the network and stored on the server in encrypted format.

EzFs also allows users to transmit and receive NTFS encrypted files over email and instant messenger. The main advantage of using EzFs over email/IM client encryption is that the data is never stored anywhere during the entire data transfer in plain text. The data is copied from the source storage device in encrypted format and remains encrypted until it is imported on the receiver storage device for decryption and access. EzFs is unique in its ability to provide end-to-end encryption of documents over email and IM.

Detailed Description

EZFS is implemented as user mode process that registers itself as a CIFS server with the local computer and runs on as an application on the logged on user session. EZFS publishes a Netbios name of type 0x20 on the local computer. Figure 3 shows the overall system architecture and how the data is accessed via the EZFS software.

When an application accesses the path \\myezfs\share the Windows CIFS redirector will try to find and connect to the name myezfs. The CIFS redirector submits the name MYEZFS<20> to the network stack to establish a connection to the server MYEZFS.

Once a connection is established with the CIFS redirector the EZFS application validates the connecting user credentials as defined by the CIFS protocol specification. The EZFS application authenticates the CIFS redirector user against the logged on user context by using the Windows security APIs InitahzeSecurityContext and AcceptSecurityContext.

Note, the EZFS doesn't require the logged on user password since this information is fully abstracted by the Windows security subsystem via the mentioned APIs.

Once an authenticated user session is successfully established between the CIFS redirector and the EZFS application the CIFS redirector sends and receives file system requests as defined by the CIFS protocol. The EZFS application implements a CIFS service that decodes and encodes CIFS packets and maps the CIFS requests to file system operations.

The virtual file system module of the EZFS application implements a simple file system and maintains its state in memory. The EZFS pseudo file system is layered on top of an existing file system and tunnels the CIFS redirector requests to the underlying file system via W1N32 API. The EZFS file system extends the underlying file system by bridging Windows EFS functionality from a local NTFS volume to the underlying file system by caching the file content in the local NTFS volume and copying the file raw encrypted data blocks from the NTFS volume into the underlying file stored in the target volume.

The client CIFS redirector and applications are unaware of this data caching and transformation as the entire process is totally transparent. EZFS uses Win32 backup APIs to copy raw data file blocks between the cached NTFS files and target files.

Another unique feature of EZFS is the ability to specify an additional EFS certificate to be inherited from the EzFs exported share and added to all files within the scope of the EzFs virtual namespace.

EZFS also provides transparent compression of data blocks using the gnu-zip (gzip) file format. The gzip file format was used because it supports per-file compression and widely supported by many other applications. EZFS however extends the gzip file format in a transparent way that is fully compatible with other decoder according to the gzip specification but improves it to support random data access. This allows EZFS to be very efficient in handling large compressed files and applications that have random JO access patterns without having to wait for the entire file to be decompressed before the first data block is available.

The virtual EZFS file system maintains a table of open files for each connected user and a table of per-file records that's global to the EzFs exported share. Each open file has a file handle instance data structure associated with it that holds the following information: -Physical file handle on physical folder being managed via EzFs -this handle can be a handle to a remote server or local flash drive obtained via CreateFile

API

-Cached file handle on local NTFS volume -this handle is obtained via CreateFile to a local file on the user's profile -Pointer to a file record -Desired access -Share access -Flags -bitmask of state values o MODIFIED -file was modified through this handle o TUNNEL -cached file handle is valid o ENCRYPTED -cached file handle is encrypted and file should be written back to physical file in encrypted format o COMPRESSED -file should be written back in compressed format or physical format is in compressed format The other important data structure is the file record. Each file that is currently being accessed has a corresponding file record. It consists of the following fields -Number of open handles using this record -Number of write handles using this record -Physical file attributes -Physical file name -Cached file name -Cached file handle -can be NULL in which case the system uses the open file handle instance copy of the cached file handle.

-Flags The following flow charts outline the basic algorithms used by the EzFs file system.

Method: CreateFile Flow diagram 1 Input: filename, file attributes, disposition, desired access and create options Output: file handle Lookup filename in global file record table If file record is not found then create a new record Create file handle instance and link it to file record Issue a CreateFile on physical file path.

If call fails then free up allocated resources and fail request Else; Remove any state in tunnel cache for filename Query physical file attributes Update file record attributes If the file represents a directory then return the created file handle instance identifier If file is encrypted by EzFs (check for system attribute) then If file record cached filename is empty then CreateorLookup remote filename in local cache Set cached filename in file record Endif If file record count == 0 file record attributes!= remote attributes then Import physical file in local cache endif Open cached file and set handle in file instance record Endif Hdl = get cached file handle if valid or remote file handle otherwise If (file is compressed by EzFs (check for alternate stream $EZFS or hidden attribute) then Read $EZFS header block to determine uncompressed file size If file is compressed and valid then Mark file instance as compressed Create compression context for handle instance Update file attributes size to uncompressed file size Endif Endif Return file instance identifier and attributes Method: ReadFile Flow diagram 2 Input: File instance identifier, buffer, length, offset, number o bytes to read Output: fill buffer with data and return number of bytes read Lookup file identifier in file instance table Hdl = file instance locally cached file handle If (1-Idi is INVALID) then Hdl = file instance physical file handle endif Hdl = Cached file handle if valid otherwise Physical file handle if (file instance is compressed) then Read compressed block by first translating offset into compressed offset Issue read operation to read compressed data from Hdl Decompress data into buffer Else Issue read operation to read data from Hdl Endif Return length of data read Method: WriteFile Flow diagram 3 Input: File instance identifier, buffer, length, offset, number of bytes to write Output: number of bytes written Validate File instance identifier Hdl = file instance locally cached file handle If (Hdl is IN VALID) then Hdl = file instance physical file handle endif If (file instance is compressed) then Lock file record If (file instance is compressed) then Decompress entire file to prepare for update Endif Unlock file record Endif issue write operation to write data to Hdl Mark file instance as MODIFIED Return length of data written Method: QueryHandleA ttributes Flow diagram 4 Input: File Instance handle, information level, buffer, length Output: fill buffer with attributes and number of bytes available Lookup File Instance Handle in file instance table Hdl = file instance locally cached file handle If (Hdl is IN VALID) then Hdl = file instance physical file handle endif Issue query file attributes on Hdl If (file instance is compressed) then If (information level requires file attributes) then Update file attribute with FILE_ATTRIBUTE_COMPRESSED endif If (information level requires file size) then Update file size Endif Endif Return file attributes Method: QueryFileNameA ttributes Flow diagram 5 Input: file name, information level, buffer, length Output: fill buffer with attributes and number of bytes available Lookup file record in global file record table If file record is found and local cached file is open then Issue query to local NTFS volume on cached file name Else Issue query to physical volume or remote server Return file attributes Method: DeleteFile Flow diagram 6 Input: filename Output: status If file is encrypted then Capture encryption context including certificates Store encryption state in memory-based tunnel cache endif Issue delete operation on physical filename If success then Remove local cached fiie Endif Method: RenameFile Flow diagram 7 Input: old filename, new filename Output: status Issue rename operation to physical volume or remote server If (success) then Issue rename on local cached file to new name in cache Lookup new name in memory-based tunnel cache If found then Propagate encryption context to renamed file Clear tunnel cache entry endif Endif Method: ListD!rectory Flow diagram 8 Input: directory name to query, information level, buffer, length Output: status, number of available bytes Issue List directory on physical volume or remote server For each entry Lookup entry in local cache If found then Adjust size and attributes based on local cached file Endif End Return data and number of bytes Method: Close Flow diagram 9 Input: File Instance handle Output: status Validate file instance handle Hdl = cached file handle if valid otherwise physical file handle If (file instance was modified) then If (file is compressed) then Compress file either by storing compressed data in $ZFS alternate stream if encrypted or directly into the default data stream otherwise endif If (file is encrypted) then Export local cached copy to remote server or physical volume Endif endif Close cached file handle if valid Close physical file handle Release reference to file record If (file record reference count is zero) then Release file record Endif Method: LockFiIe Flow diagram 10 Input: File Instance, offset, length, exclusive flag, wait_flag Output: status Validate file instance Issue lock request on physical handle to physical volume or remote server If (operation is not supported or success) then Issue local request on cached handle Return status; Endif Return status; Method: UnlockFile Flow diagram 11 Input: File Instance, offset, length, exclusive flag, wait_flag Output: status Validate File Instance Issue unlock request to physical handle to physical volume or remote server If (operation is not supported or success) then Issue unlock request on cached handle Return status Endif Return status Encrypted Files: EzFs uses the Win32 file encryption APIs to export and import the encrypted blocks of the tile. Encrypted files are stored on the physical volume as defined by these APIs.

EzFs uses the system bit to mark encrypted files written in Windows backup format. This avoids the need to read and check every file for encryption. The system bit is a hint since unencrypted system files will have this bit set but don't conform to the EFS format and EzFs will treat them as normal files in such case. The system bit was selected because it avoids double encryption of files encrypted by EZFS that are stored on NTFS volumes since Windows EFS ignores files set the system bit set. In addition the system bit is available and supported by all files systems and not visible to users by the Windows Explorer UI which reduces the risks of users manipulating the system bits on files stored in EZFS managed folder.

EzFs makes it possible for users to use NTFS EFS for protecting mobile and network data in a simple and transparent manner. EzFs also supports adding and removing users in much the same manner as local NTFS EFS files.

Many applications update files by first creating a new temporary files and renaming the temporary files on the original files. This process can result in loss of encryption context if the application is not careful to preserve and duplicates the encryption context of the original files to the temporary files. EzFs maintains an encryption context tunnel cache across file delete and rename operations to handle such applications. The tunnel cache works by capturing and storing in memory a file encryption state before deletion or rename and storing it for a short period of time in memory. If a file with this name is recreated shortly before the tunnel cache entry is timed out the cached encryption context is applied to the new file and cleared from the cache. In this manner encrypted files that are modified on different user context using different certificates preserve the original certificates value on the file prior to modifications without depending on the application to preserve the context across updates.

Compressed Files: EZFS implements a 64 Khytes block based compression algorithm in order to efficiently support random read on large files. Because only 64 Kbytes blocks are decompressed at any given time EZFS doesn't need to decompress the entire file to provide random read access to the underlying file. This is important in scenarios where only specific blocks in the file are read. For example, an application might want to extract and display extended properties or icon information and doesn't require the file to be decompressed in its entirety. EZFS uses gzip file format and the random access extended header format.

Similar to encrypted files, EZFS overloads file system attributes to mark compressed files. This avoids reading all files to check for compression. The hidden file attribute is used to mark EZFS compressed file. The selection of the hidden bit was based on the fact at all file systems have native support of it.

Methods Summary

I/O model -Encrypted files maintain two handles -local and remote -File Locking is preserved and applied on both handles -File Attributes are updated on both handles -Query attributes are applied on the local file only -Query directory retrieves the remote directory and updates entries locally cached -Files not locally available are marked with offline bit to avoid unnecessary recall of files -Local cached files are native NTFS files but physical files are backup EFS files -Compressed EFS files use an alternate stream to store the zippedlcompressed Encryption -EzFs enables NTFS EFS files to be stored compressed and encrypted on disk or other storage media -Client-side encryption based on local NTFS and data read/written over the network in encrypted format. Data may be stored on the remote server as a blob or native NTFS EFS encrypted file -Use system-bit to mark ezfs files as system-bit is not encrypted by default by NTFS which avoids double encryption of already encrypted files. This also enables EzFs to only try to restore files with system bits and avoid the overhead of reading non-EFS files.

-On create or read requests EZFS restores files to the local volume before completely process the create/open operation Compression -Transparently compress and decompress data blocks inline with user access -Support random 10 access of compressed data without the need to decrypt the entire file before first access Offline Files Integration Figure 4 shows how EzFs extends offline folders to cache data from different storage devices besides CIFS network servers. It allows Offline folders to cache content from Flash Media, CDROM, local fixed or removable drives, Web, FTP or NFS server. This enabled users to work offline against non-CIFS servers and removable drives when disconnected and have continuous access to their documents and folders. Furthermore, since EZFS integrates itself seamlessly with the Windows environments it can leverage the existing user experience and Explorer interfaces to manage the offline content. More importantly, because Offline Folders is built into the Windows operating system users can leverage the synchronization manager functionality to manage their local Offline Folder. This eliminates the need to manually manage and maintain multiple copies of the data to gain offline access. The ability to cache data stored and accesses via nonCIFS servers in the Offline Files is unique to EzFs and enables users to consolidate documents stored in multiple storage platforms into a single Offline Files cache.

When Offline Files are combined with EZFS Windows EFS encryption it provides an Important security features to protect the Offline Files content. Figure 2 shows how users can use EZFS to protect content in the local Offline Files cache and remote CIFS file server. Although Windows XP supports EFS Offline Files encryption it does so in the system context and not the user context which is less security. User-based encryption of the Offline Files cache via EZFS also enables users to share encrypted files when disconnected.

Secure Copy/Paste and Email Attachments EzFs allows users to create encrypted copies of files by extract files in encrypted format into other files. The encrypted version of the original files can further be shared with other users via EMAIL, Instant Messaging, CDROM, FLASH DRIVES, network shares or local folders -see figure 5. During the creation of the encrypted files EzFs allows users to add or remove additional encryption certificates to enable sharing of encrypted files among different users. This functionality and flexibility allow EZFS to become a truly mobile or transportable encrypting file system platform for all uses. It also allows administrators to deploy a single encryption technology through their environment and continue to protect the data via the same mechanism independent of where it is stored or how it is transmitted.

EZFS "Secure Copy" feature interacts with the Windows Explorer Shell via the Clipboard to store a path to the encrypted file of the file the user copied. The encrypted file is stored in the user's profile temporary files folder and its path stored in the Clipboard.

When a user invokes the "secure copy" menu context on a selected file the EzFs application is instantiated by the Windows Explorer. The EzFs application creates a temporary encrypted file in the user's temporary files folder and copies the content of the selected file into the temporary encrypted file to produce an encrypted version of the original file. The EzFs application then displays a user interface to allow the user to add or remove certificates that can decrypt the encrypted file content. Once the user have selected the set of certificates the EzFs uses Win32 EFS APIs to add any additional certificates and to export the raw encrypted blocks and copy it to a secondary temporary file with the name of the original selected file with an additional special EzFs file extension. The user can then select the Paste command on the Windows Explorer to insert the encrypted file in a folder, email attachment or IM session.

When a file with the special EzFs file extension is selected the Windows Explorer is invoked and the EzFs application is dispatched to handle the file selection. The EzFs application creates a temporary encrypted file in the user's temporary folder and uses the Win32 EFS APIs to import the raw encrypted data blocks in the created temporary file.

The temporary file name consist of the original file name with the special EzFs file extension stripped. The EzFs application then invokes the Windows Shell Execute API to transfer control the Windows Shell.

Certificate Management Creating transportable Windows EFS files is not sufficient to enable sharing of encrypted content between different users. Windows EFS uses a private/public key infrasucture to protect the file encryption key. EZFS simplifies the certificate management complexity by allowing users to create EFS certificates dynamically and to assign them to different users, devices or folders.

The ability to dynamically create EFS certificates is critical to support mobile EFS encryption on Flash Drives. EZFS allows users to generate EFS certificates and assign them to the Flash Drive. Each EZFS application instance can be configured with (an) extra EFS certificate(s) to be added to all files as they are created and updated. EZFS also transparently imports and exports the device certificate in and from the user store to provide a searnJess EFS file access user experience. Certificates on the device are secured with a user-specified password or a hash of the user's log on password to provide a single sign-on experience. (4

Claims

Potential Claims 1-A mechanism to transparently secure files on volumes

that lack encryption by using Windows NTFS EFS on local drive as a temporary file cache.

2-A mechanism to transparently Import and export files from one volume to a secondary volume that implements and supports features lacking the former volume.

3-A mechanism to transparently maintains file handles to a physical file and a local temporary cached file and redirect JO requests to one or both file handles.

4-Assign Flash Drive or removable media a network identity (server name or IP V4/V6 address) and UNC path to make device universally accessible over local area network or Internet.

5-Flash Drive retains its UNC path identity independent of computing unit it's attached to.

6-A mechanism to create and assign certificate to Flash Drive, removable media or folder to enable mobility and group sharing.

7-A mechanism to import Flash Drive certificate on device plug-in and removal of certificate when device is unplugged.

8-The ability to transparently tunnel encryption certificates of files across file rename operations.

9-A mechanism to provide folder level certificate inheritance for encrypted files and the ability to add inherited certificate to saved and created files transparently.

10-A mechanism to copy files from any folder as an encrypted EFS blob and store the encrypted raw data into a second file that contains the encrypted data and its name tagged with a special extension.

11-A mechanism to send and receive files in encrypted Windows NTFS EFS backup format via email, instant messaging, file transfer protocol, HTTP or other remote data transfer protocols.

12-Transparent caching of remote content from non-CIFS servers in the Offline Files local hard drive cache 13-Transparent caching of remote content access from non-CIFS servers via protocols that have different semantics, authentication models, JO model, non-network like CDROM or Flash Drive that are not CIFS servers.

14-A gateway mechanism to translate CIFS protocol into other protocols without loss of application semantics including but not limited to WebDAV (Internet Distributing Authoring and Versioning) over SSL (Secure Socket Layer) to Windows SharePoint Services or Web Server, FTP (File Transfer Protocol) server and NFS (Network File System) server to enable transparent file access to remote content as CIFS servers and caching of remote content in Offline Files local drive cache.

15-The ability to create a UNC name alias to physical path in URL, FTP and other naming formats where the UNC name doesn't contain parts of the physical path.

I6-Transparent caching of content from removable media in the Offline Files local hard drive cache.

IC