u, -update OLD-DB File name old databaseĬhecksums in the input database named OLD-DB are used for If -o is missing, then a name is constructed using theĭate and the strftime template '%Y%m%d%H%M%S_dircsum.sqlite'. o, -output OUT-DB output database File name | | 1) Code for why csum performed, 2) regular | | | | Print a record for each csum performed: | | | 16 | Super resolution CSUM progress | 2 8 | | | Print a code for why each csum was done | | | 8 | High resolution CSUM progress | 2 16 | | 4 | High resolution Scan progress messages | 32 | | 2 | DB & CSUM write progress bar | 8 16 | | 1 | Low resolution progress messages | | Scan the directory tree rooted at, and create an SQLite3 database For me the more useful application is found in the dcsumCmpDB.rb andĭcsumCmp.rb scripts. Probably the most popular use is to do things like auditįile types stored on file servers. It scans a directory hierarchy and stores file-system metadata in an SQLite database.įor many end users this DB and the data it provides is the ultimate end goal for using this code. The main data collection script is dcsumNew.rb. How much data is owned by people no longer employed at my company?.How much space can we save if we move stuff not modified in 6 months to cold storage?.I need to know my data churn rate so I can compute my snapshot storage requirements.How much space would we save if we switched from TIFF to PNG as our standard image format?.If we enabled dedup, how much space would we save?.Probably the most common application people write to me about is file server usage pattern analysis. Verify the integrity of my " static" backups Just to make sure my weekly & monthly full backups really are good – without testing them via a full restore. Security or privacy of the cloud provider's encryption. This is a nice way to back up stuff to the cloud without depending on the Drive my " dynamic" backup scripts This system makes snapshots of what's changing in my working trees just in case I fat finger something beyond git's ability to recover Drive my cloud sync scripts I sync encrypted files to the cloud using the content hash as the filename. Track file-system changes over time Mostly to help me plan for future disk purchases and size my backup needs over time. These tools provide a way to collect file-system metadata, store that metadata into an SQL database, and then conveniently query that data or compareĭatabases. Shell, org-mode file, and sqlite3 (with a bit of ruby) Materialize the BIG view and add indexes to result ( faster3) Materialize smaller views and add indexes to result ( faster2) Add indexes for faster queries ( faster1) Megaabytes/second processed durring DB dump and CSUM Objects/second process for entire process run time Objects/second scanned during filesystem scan phase Fast method using lft from the fsobj table Fast method using id from the fsobj table If id is in pre-order tree traversial order, then… Objects with same name but diffrent time stamps Objects with same name but diffrent content Checksums that only exist in one of the scans Files on right with content matching no file on left Files on left with content matching no file on right Size of the objects directly contained in that directory Disocver the lft and rgt values for our path By each user ( userid, uname, =groupid & gname) variance, mean, count, sum, & sum of squares broken out by file extention variance, mean, count, sum, & sum of squares Relative path and size for the median file Relative path and size for the largest 20 files Relative path and size for the largest file(s) List check sum and fully qualified path names for non-empty, regular files with the same check sums Space consumed by files which have no owner in the groups table Fully qualified path names for files which have no owner in the groups table Space consumed by files which have no owner in the users table Fully qualified path names for files which have no owner in the users table Select objects directly in this sub-directory Find all roots of all subtrees containing object Selecting objects in a sub-tree or sub-directory
0 Comments
Leave a Reply. |