Cloud Storage

Cloud Storage stores objects in buckets.

There are several differences between Cloud Storage and a file system.

1. A file system has a hierarchical structure. Cloud Storage is unstructured. It is a flat system of buckets (not directories) that cannot be nested.

2. An object name may consist of up to 222 characters. A valid character in an object name include '/' (forward slash). Using this character in object names can simulate some of the hierarchical structure of a file system, even though the slash is not a functionally significant entity.

3. Objects are replicated and distributed for availability. However, there is no distributed equivalent of a file lock. Therefore, the last entity to write to an object "wins." If you use Cloud Storage in a distributed application, the application is responsible for locking and serialization of access.

4. Cloud Storage treats objects as an unstructured series of bytes.

Multi-Regional = Data is stored redundantly in multiple locations

Regional is typically lower cost than Multi-Regional. When you select Regional, you must choose a location.

Nearline and Coldline storage incur an "early deletion" charge of the minimum number of days, if an object is deleted before that time.

Regional and Multi-Regional storage classes typically return the first byte in less than a second (and often in tens of milliseconds). Nearline and Coldline storage may take seconds before the first byte is retrieved.

gsutil command to create bucket

gsutil mb [-b <on|off>] [-c class] [-l location] [-p proj_id]

          [--retention time] url...

Example:

gsutil mb -c nearline gs://some-bucket

You can specify one of the available locations for a bucket with the -l option.

Examples:

gsutil mb -l asia gs://some-bucket

gsutil mb -c standard -l us-east1 gs://some-bucket

Bucket Name requirements:

● globally unique

● lowercase, #s, -, . (3-63 chars)

● URI = DNS CNAME

● Access control

○ ACL

○ Signed access

● Encryption at rest

● No minimum size; unlimited storage

● Pay for use

● 99.999999999% durability

● Low latency (time to first byte is typically tens of milliseconds)

● API access

The dot "." is also a valid character in a bucket name. It can be used to create domain-named buckets, such as mybucket.example.com. However, there is a verification process required to prove that you are the owner of the domain before this kind of bucket can be created. Max URI bucket name is 222 characters, with max 63 characters between dots.

Retention Policy

You can specify retention period in one of the following formats:

--retention <number>s

Specifies retention period of <number> seconds for objects in this bucket.

--retention <number>d

Specifies retention period of <number> days for objects in this bucket.

--retention <number>m

Specifies retention period of <number> months for objects in this bucket.

--retention <number>y

Specifies retention period of <number> years for objects in this bucket.

Examples:

gsutil mb --retention 1y gs://some-bucket

gsutil mb --retention 36m gs://some-bucket

If you don't specify a --retention option, the bucket is created with no retention policy.

Bucket Policy Only

You can specify one of the available settings for a bucket with the -b option.

Examples:

gsutil mb -b off gs://bucket-with-acls

gsutil mb -b on gs://bucket-with-no-acls

Change the default Storage Class

·         You can change the default storage class of a bucket.

·         The default class is applied to objects as they are created in the bucket.

·         The change only affects new objects added after the change.

·         A Regional bucket can never be changed to Multi-Regional.

·         A Multi-Regional bucket can never be changed to Regional.

·         Objects can be moved from one bucket to another bucket with the same storage class from the GCP Console; however moving objects to buckets of different storageclasses requires using the gsutil command from CloudShell