Greg Glidden asked the following question on the ILTA-DESKTOP forum:
“It can be a fairly common occurrence for a firm to have to store, share, and manage various types of large files. How do you deal with large, non-standard data types such as audio, video, or ISO files? Are there special considerations for ECM? What issues have you run into and what solutions have worked best? What tools do you use; file share, database, O365 Video, Vimeo, YouTube, etc.?”
My reply – which addresses the storage part of his question:
“We tell users to make a folder with the client-matter name on a network share (Large Client Files)X: and save files there that are too big to save into iManage. They are supposed to make a document in the iManage client-matter workspace with a file name that essentially says "Additional Large files can be found on X:." Since these backups are seldom if ever accessed, we back them up to Amazon Glacier using the Cloudberry Client. Glacier is dirt cheap and slow as molasses if you need to do a restore but we've got the Large Files share backed up for an affordable price without breaking the budget since we don't include it with main production backups.”
To explain a bit further, in order to have a place on our network that is backed up where users can store copies of large files that won’t go into our DMS, we added a volume to our file server and made it available to them as drive letter (Large Client Files)X:. This drive is only backed up locally with our main Storagecraft backups as replicating to our Storagecraft cloud account at $1 per GB along with our other backups would have been costly (we currently have about 1TB stored on X: ). I’d heard that Amazon Glacier is an extremely affordable, secure way to backup data to the cloud that seldom needs to be touched so I checked into it and ended up using it as our cloud backup for this “Large Files” share. Here is a quick summary:
Pricing
Pricing is based on activity and is a bit confusing, but the bottom line is the more activity (upload/download/read/write/change), the more it costs. Also, the slower the speed you are willing to accept to retrieve data, the cheaper it gets. If you just want to upload data one time and never add to it or access it, cost to upload is zero and storage cost in AWS’s US locations is $.004 - $.005/GB/mo. We are storing 1.1 TB and do a nightly incremental – which often has zero bytes – and have been paying about $4.50/mo.
When setting up an Amazon Glacier vault there is an option to select a retrieval policy tier – “Free,” “Max Retrieval Rate” (enter a transfer speed in GB per hour and it tells you the rate), or “No Retrieval Limit.” I picked the first option which means we could retrieve up to 10 GB /mo for free.
Amazon provides a number of online tools to help understand and estimate future costs. A complete breakdown and explanation of pricing can be viewed at https://aws.amazon.com/glacier/pricing/
Security
Amazon Glacier uses 256-bit Advanced Encryption Standard (AES-256) and by default manages the keys. According to the Glacier FAQ “Customers wishing to manage their own keys can encrypt data prior to uploading it.” Stated policy is “We do not disclose customer content unless we're required to do so to comply with the law or a valid and binding order of a governmental or regulatory body.” Also “…our customers can encrypt their customer content, and we provide customers with the option to manage their own encryption keys.” Our data stored in the cloud is encrypted by choosing 256-bit encryption and entering a password using the Cloudberry Client mentioned below. I’d highly recommend doing this so that AWS does not have the ability to provide your data to anyone if ordered to do so.
Amazon Glacier includes a number of options to enforce compliance such as “write once read many” (WORM) allowing data to be uploaded once, viewed whenever needed but not changed.
NOTE: When creating your Amazon Glacier account, be sure to save the Access Key ID and Secret Access Key someplace secure where you can retrieve it later as it will be needed to set up a backup job.
Access
In order to upload/download/view data to/from Amazon Glacier, you’ll either need to interact using the AWS Command Line Interface (CLI), write code to make requests using the REST API or the AWS SDKs, or buy a third party client program.
I went with the third option and bought the program “Cloudberry Backup Server Edition” for $119.99 (see
https://www.cloudberrylab.com/backup/windows-server.aspx ). This program works with 59 different cloud storage providers including Amazon
Glacier.
Installation is straight forward – just keep clicking “Next” and enter your name and the product key provided in the confirmation email received after buying the program. To create a backup job choose “Local to Cloud” -> Amazon Glacier -> enter a Display Name for the backup and enter your Glacier access key and secret key recorded earlier. Choose the vault created from the drop down list and keep going through the backup wizard. Be sure to choose AES-256 for encryption, enter a password and be sure to also save it in a secure place as it will be needed to decrypt any backup restores. Choose any other desired options including backup frequency, email notifications for success/failure, etc. - until completed.
To do a restore, click the Backup Storage tab, browse the folder structure of your Amazon Glacier Vault, right click on the desired file(s) or folder(s), choose Restore and follow the wizard. You’ll be prompted to enter encryption the password used when the backup job was set up. NOTE: Living up to the product’s name - “Glacier” - using the default restore option, it can take 3-5 hours to complete even the smallest restore job.
That’s it!
FAQ’s for the Cloudberry Backup can be accessed at
https://www.cloudberrylab.com/backup-faq.aspx FAQ’s for Amazon Glacier are here
https://aws.amazon.com/glacier/faqs/ A guide for getting started is available at
https://docs.aws.amazon.com/amazonglacier/latest/dev/amazon-glacier-getting-started.html Hopefully this write-up helps someone out!
Tom Esker
IT Manager
Wyatt Early Harris Wheeler LLP
High Point, NC