2 years agoRemove Permission whitespace workaround master
thobrla@google.com [Thu, 21 Feb 2013 21:09:07 +0000 (21:09 +0000)]
Remove Permission whitespace workaround

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@346 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed two version-specific URI bugs:
mfschwartz@google.com [Thu, 21 Feb 2013 19:26:32 +0000 (19:26 +0000)]
Fixed two version-specific URI bugs:

1. gsutil cp -r gs://bucket1 gs://bucket2 would create objects in
bucket2 with names corresponding to version-specific URIs in bucket1
(e.g., gs://bucket2/obj#1361417568482000, where the "#1361417568482000"
part was part of the object name, not the object's generation).

This problem similarly caused gsutil cp -r gs://bucket1 ./dir to create
files names corresponding to version-specific URIs in bucket1.

2. gsutil rm -a gs://bucket/obj would attempt to delete the same object
twice, getting a NoSuchKey error on the second attempt.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@345 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRelease 3.24
mfschwartz@google.com [Wed, 20 Feb 2013 00:56:03 +0000 (00:56 +0000)]
Release 3.24

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@344 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFix bug that caused attempt to dupe-encode a unicode filename
mfschwartz@google.com [Wed, 20 Feb 2013 00:01:54 +0000 (00:01 +0000)]
Fix bug that caused attempt to dupe-encode a unicode filename

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@343 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFix flaky tests.
jterrace@google.com [Tue, 19 Feb 2013 23:38:43 +0000 (23:38 +0000)]
Fix flaky tests.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@342 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years ago- Refactored retry logic from setmeta and chacl to use @Retry decorator.
zwilt@google.com [Tue, 19 Feb 2013 19:39:32 +0000 (19:39 +0000)]
- Refactored retry logic from setmeta and chacl to use @Retry decorator.
- Moved @Retry decorator to third_party.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@341 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRelease 3.22
mfschwartz@google.com [Fri, 15 Feb 2013 19:49:11 +0000 (19:49 +0000)]
Release 3.22

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@340 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoUpdated copyright notice on code files
mfschwartz@google.com [Thu, 14 Feb 2013 20:03:00 +0000 (20:03 +0000)]
Updated copyright notice on code files

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@339 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoImplement support for gsutil cp file -
mfschwartz@google.com [Thu, 14 Feb 2013 19:46:51 +0000 (19:46 +0000)]
Implement support for gsutil cp file -

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@338 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoClean up/clarify/deepen various parts of gsutil built-in documentation
mfschwartz@google.com [Thu, 14 Feb 2013 19:38:09 +0000 (19:38 +0000)]
Clean up/clarify/deepen various parts of gsutil built-in documentation

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@337 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixes chacl help text to be consistent with actual syntax
bensonk@google.com [Wed, 13 Feb 2013 23:59:32 +0000 (23:59 +0000)]
Fixes chacl help text to be consistent with actual syntax

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@336 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoUpdated testing infrastructure.
jterrace@google.com [Wed, 13 Feb 2013 22:36:46 +0000 (22:36 +0000)]
Updated testing infrastructure.

Cleaned up last remaining old tests:
 * Overhauled test_commands.py to use the new testing infrastructure and renamed it.
 * Overhauled test_wildcard_iterator.py to use the new testing infrastructure.

Windows bug fixes:
 * Fixed various places from a hard-coded '/' to os.sep.
 * Fixed a bug in the cp command where it was using the destination URI's .delim
   property instead of the source URI.
 * Fixed a bug in the cp command's _SrcDstSame function by simplifying it to use
 * Fixed windows bug in tests/util.py _NormalizeURI function.
 * Fixed ZeroDivisionError sometimes happening during unit tests on Windows.

Testing improvements:
 * Moved TempDir functions from GsUtilIntegrationTestCase to GsUtilTestCase.
 * Made test runner message show the test function being run.
 * Added file path support to ObjectToURI function.
 * Disable the test command if running on Python 2.6 and unittest2 is not
   available instead of breaking all of gsutil.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@335 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoReworked gsutil commands to work with version-specific URIs that embed only generatio...
mfschwartz@google.com [Wed, 13 Feb 2013 20:53:48 +0000 (20:53 +0000)]
Reworked gsutil commands to work with version-specific URIs that embed only generation, not meta-generation

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@334 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded chacl command module and relevant tests.
bensonk@google.com [Wed, 13 Feb 2013 02:20:36 +0000 (02:20 +0000)]
Added chacl command module and relevant tests.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@333 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agodded retry logic to setmeta's optimistic concurrency approach.
zwilt@google.com [Wed, 13 Feb 2013 01:14:28 +0000 (01:14 +0000)]
dded retry logic to setmeta's optimistic concurrency approach.

See https://codereview.appspot.com/7311079/.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@332 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRemove XML parsing from setacl
thobrla@google.com [Mon, 11 Feb 2013 23:31:43 +0000 (23:31 +0000)]
Remove XML parsing from setacl

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@331 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoImplemented gsutil cp -v option for printing the version-specific URI that was just...
mfschwartz@google.com [Sun, 10 Feb 2013 18:59:28 +0000 (18:59 +0000)]
Implemented gsutil cp -v option for printing the version-specific URI that was just created

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@330 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoReworked versioned StorageUri and version handling in gsutil:
mfschwartz@google.com [Fri, 8 Feb 2013 00:25:47 +0000 (00:25 +0000)]
Reworked versioned StorageUri and version handling in gsutil:

- Moved StorageUri parsing that had been split across storage_uri and conveience wrapper into storage_uri, so parsing works uniformly from everywhere.
- Removed need for commands to specify -v option to parse versions. Versioned URIs are now uniformly recognized by all commands.
- Rewrote versioning help
- Fixed bug where ACL setting code didn't pass meta_generation through call layers, so didn't correctly work with versioned objects
- Implemented setacl integration tests

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@329 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoPass headers with API version & project ID to create_bucket().
yovadia@google.com [Thu, 7 Feb 2013 16:49:12 +0000 (16:49 +0000)]
Pass headers with API version & project ID to create_bucket().

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@328 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed bug in copying subdirs to subdirs.
jterrace@google.com [Wed, 6 Feb 2013 19:36:28 +0000 (19:36 +0000)]
Fixed bug in copying subdirs to subdirs.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@327 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoUse GCS V2 API and project_id from boto config if necessary in integration_testcase...
yovadia@google.com [Tue, 5 Feb 2013 18:20:48 +0000 (18:20 +0000)]
Use GCS V2 API and project_id from boto config if necessary in integration_testcase#CreateBucket().

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@326 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded timestamp of test measurement in perfdiag command.
jterrace@google.com [Mon, 4 Feb 2013 17:22:18 +0000 (17:22 +0000)]
Added timestamp of test measurement in perfdiag command.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@325 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoimplemented test for gsutil cp -D
mfschwartz@google.com [Sun, 3 Feb 2013 23:29:33 +0000 (23:29 +0000)]
implemented test for gsutil cp -D

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@324 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoImplemented gsutil option for copying between locations and storage classes
mfschwartz@google.com [Sun, 3 Feb 2013 05:39:56 +0000 (05:39 +0000)]
Implemented gsutil option for copying between locations and storage classes

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@323 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoTest var renaming to match typing
mfschwartz@google.com [Sun, 3 Feb 2013 03:06:43 +0000 (03:06 +0000)]
Test var renaming to match typing

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@322 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFix ascii codec decode error when constructing tracker filename from non-7bit ASCII...
mfschwartz@google.com [Sat, 2 Feb 2013 19:40:46 +0000 (19:40 +0000)]
Fix ascii codec decode error when constructing tracker filename from non-7bit ASCII intput filename

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@321 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded error detail for attempt to delete non-empty versioned bucket. Also added versi...
mfschwartz@google.com [Sat, 2 Feb 2013 04:08:05 +0000 (04:08 +0000)]
Added error detail for attempt to delete non-empty versioned bucket. Also added versioning state to ls -L -b gs://bucket output

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@320 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded gsutil help text for common 403 AccountProblem error
mfschwartz@google.com [Fri, 1 Feb 2013 23:09:04 +0000 (23:09 +0000)]
Added gsutil help text for common 403 AccountProblem error

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@319 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRefactored gsutil tests.
jterrace@google.com [Wed, 30 Jan 2013 19:02:47 +0000 (19:02 +0000)]
Refactored gsutil tests.

This removes the test_steps property from the Command class. Instead, each
command now has its own test file in gslib/test/, e.g. gslib/commands/cp.py
corresponds with gslib/tests/test_cp.py. This enables a more powerful testing
infrastructure than the previous commands, makes the tests more readable, and
enables better debugging output when tests fail.

I also added a small subclass of TextTestRunner that shows a progress indicator
when running tests. The "gsutil test" command also has two new parameters, -u
that runs *only* unit tests (so it runs quickly) and -f (failfast) that will
fail immediately when the first failure happens.

The tests are currently broken on Windows. Rather than making this change even
bigger, I thought it would be better to submit a followup change to fix Windows.
All of the tests are passing on Ubuntu and OS X.

Also fixed the getcors and getwebcfg commands not to mix stderr with stdout so
that redirection to file works propertly and removed duplicate newlines.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@318 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed unit tests by using a GS-specific mocking class to override the S3
jterrace@google.com [Mon, 28 Jan 2013 16:40:31 +0000 (16:40 +0000)]
Fixed unit tests by using a GS-specific mocking class to override the S3

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@317 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoChanged cross-provider copies to use resumable or one-shot uploads instead
mfschwartz@google.com [Mon, 28 Jan 2013 02:37:04 +0000 (02:37 +0000)]
Changed cross-provider copies to use resumable or one-shot uploads instead
of streaming uploads, as appropriate. Previously, for transfers into GCS
we used streaming uploads (which weren't resumable so could be wasteful
if a failure occurred partway through a large transfer); and for transfers
into S3 we used buffering through a local temp file (which had scaling and
performance problems).

Also, moved KeyFile class to boto, and updated documentation for perfdiag

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@316 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoDefine IsLatest() for bucket_listing_refs representing local files.
yovadia@google.com [Fri, 25 Jan 2013 02:52:12 +0000 (02:52 +0000)]
Define IsLatest() for bucket_listing_refs representing local files.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@315 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoVersioning bug fixes, performance improvements.
yovadia@google.com [Wed, 23 Jan 2013 15:37:24 +0000 (15:37 +0000)]
Versioning bug fixes, performance improvements.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@314 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoChange Command.Apply method to be resilient to name expansion exceptions.
jterrace@google.com [Wed, 23 Jan 2013 01:28:47 +0000 (01:28 +0000)]
Change Command.Apply method to be resilient to name expansion exceptions.

Before this change, if an exception was raised during iteration of
NameExpansionResult, the parent process would immediately stop execution,
causing the _EOF_NAME_EXPANSION_RESULT to never be sent to child processes. This
resulted in the process hanging forever.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@313 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoMoved gslib/test_*.py unit tests to gslib/tests module.
jterrace@google.com [Wed, 23 Jan 2013 00:09:56 +0000 (00:09 +0000)]
Moved gslib/test_*.py unit tests to gslib/tests module.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@312 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoChange gsutil mv to allow moving local files/dirs to the cloud
mfschwartz@google.com [Tue, 22 Jan 2013 20:38:19 +0000 (20:38 +0000)]
Change gsutil mv to allow moving local files/dirs to the cloud

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@311 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRemove gsutil cp -N option; minor cp doc fixes
mfschwartz@google.com [Tue, 22 Jan 2013 02:53:06 +0000 (02:53 +0000)]
Remove gsutil cp -N option; minor cp doc fixes

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@310 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded friendlier error message if test path munging fails.
jterrace@google.com [Mon, 14 Jan 2013 23:46:15 +0000 (23:46 +0000)]
Added friendlier error message if test path munging fails.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@309 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded tests for reading and seeking past end of file.
jterrace@google.com [Mon, 14 Jan 2013 23:46:10 +0000 (23:46 +0000)]
Added tests for reading and seeking past end of file.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@308 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded seek support to KeyFile and tests for it in new tests module.
jterrace@google.com [Mon, 14 Jan 2013 23:46:02 +0000 (23:46 +0000)]
Added seek support to KeyFile and tests for it in new tests module.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@307 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed reporting bits/s when it was really bytes/s.
jterrace@google.com [Sat, 12 Jan 2013 02:09:04 +0000 (02:09 +0000)]
Fixed reporting bits/s when it was really bytes/s.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@306 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded text to 'gsutil help dev' about legal agreement needed with code submissions
mfschwartz@google.com [Thu, 10 Jan 2013 01:05:43 +0000 (01:05 +0000)]
Added text to 'gsutil help dev' about legal agreement needed with code submissions

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@305 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoremove TooManyAuthHandlerReadyToAuthenticate catch from gsutil, now that
mfschwartz@google.com [Wed, 9 Jan 2013 00:00:39 +0000 (00:00 +0000)]
remove TooManyAuthHandlerReadyToAuthenticate catch from gsutil, now that
that exception no longer exists (per https://github.com/boto/boto/commit/dc39e0296eaf612a34637afff8be70168f381286)

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@304 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded SIGQUIT handler, to allow breakpointing a running gsutil
mfschwartz@google.com [Tue, 8 Jan 2013 22:37:03 +0000 (22:37 +0000)]
Added SIGQUIT handler, to allow breakpointing a running gsutil

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@303 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFix gsutil rm bug that caused exit status 1 when encountered non-existent URI
mfschwartz@google.com [Fri, 4 Jan 2013 18:57:04 +0000 (18:57 +0000)]
Fix gsutil rm bug that caused exit status 1 when encountered non-existent URI

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@302 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFix for cp test; update to 'help dev' doc; typo fix in setmeta doc
mfschwartz@google.com [Fri, 4 Jan 2013 17:50:14 +0000 (17:50 +0000)]
Fix for cp test; update to 'help dev' doc; typo fix in setmeta doc

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@301 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoUpdated doc for cp command regarding metadata not being preserved when copying betwee...
mfschwartz@google.com [Thu, 3 Jan 2013 21:58:41 +0000 (21:58 +0000)]
Updated doc for cp command regarding metadata not being preserved when copying between providers

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@300 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded gsutil cp -N option to override streaming uploads
mfschwartz@google.com [Thu, 3 Jan 2013 03:51:31 +0000 (03:51 +0000)]
Added gsutil cp -N option to override streaming uploads

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@299 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed gsutil ls command documentation typo for the -L option.
mfschwartz@google.com [Sun, 30 Dec 2012 23:29:51 +0000 (23:29 +0000)]
Fixed gsutil ls command documentation typo for the -L option.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@298 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoMade gsutil rm -r gs://bucket/folder remove xyz_$folder$ object (which is created...
mfschwartz@google.com [Fri, 21 Dec 2012 16:59:26 +0000 (16:59 +0000)]
Made gsutil rm -r gs://bucket/folder remove xyz_$folder$ object (which is created by various GUI tools).

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@297 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRemove leftover cloudauth and cloudreader dirs.
mfschwartz@google.com [Thu, 20 Dec 2012 21:31:35 +0000 (21:31 +0000)]
Remove leftover cloudauth and cloudreader dirs.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@296 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRemove cloudreader and cloudauth sample code from gsutil. Sample code now lives at...
mfschwartz@google.com [Thu, 20 Dec 2012 21:11:03 +0000 (21:11 +0000)]
Remove cloudreader and cloudauth sample code from gsutil. Sample code now lives at https://github.com/GoogleCloudPlatform

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@295 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoChanged gsutil update command to insist on is_secure and https_validate_certificates
mfschwartz@google.com [Tue, 18 Dec 2012 01:13:55 +0000 (01:13 +0000)]
Changed gsutil update command to insist on is_secure and https_validate_certificates

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@294 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded HTTP scheme to doc/examples for gsutil setcors command
mfschwartz@google.com [Mon, 17 Dec 2012 17:56:03 +0000 (17:56 +0000)]
Added HTTP scheme to doc/examples for gsutil setcors command

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@293 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed help command handling when terminal does not have the number of rows set
mfschwartz@google.com [Fri, 14 Dec 2012 00:32:53 +0000 (00:32 +0000)]
Fixed help command handling when terminal does not have the number of rows set

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@292 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoChange cp tests that don't specify a Content-Type to check for new
jterrace@google.com [Thu, 13 Dec 2012 18:12:14 +0000 (18:12 +0000)]
Change cp tests that don't specify a Content-Type to check for new
binary/octet-stream default instead of server-detected mime type.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@291 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoChange minimum version in documentation from 2.5 to 2.6 since gsutil no longer works...
jterrace@google.com [Thu, 13 Dec 2012 18:11:07 +0000 (18:11 +0000)]
Change minimum version in documentation from 2.5 to 2.6 since gsutil no longer works in Python 2.5.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@290 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdds perfdiag command and fixes Windows compatibility.
jterrace@google.com [Thu, 13 Dec 2012 00:44:15 +0000 (00:44 +0000)]
Adds perfdiag command and fixes Windows compatibility.

The perfdiag command runs a diagnostic test against a bucket, collects system
information, and reports results.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@289 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoChanged resumable upload threshold from 1 MB to 2 MB
mfschwartz@google.com [Tue, 11 Dec 2012 23:01:52 +0000 (23:01 +0000)]
Changed resumable upload threshold from 1 MB to 2 MB

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@288 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed bug where gsutil test only cleans up first few test files
mfschwartz@google.com [Tue, 11 Dec 2012 00:41:52 +0000 (00:41 +0000)]
Fixed bug where gsutil test only cleans up first few test files

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@287 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRelease 3.21
mfschwartz@google.com [Mon, 10 Dec 2012 22:17:18 +0000 (22:17 +0000)]
Release 3.21

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@286 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoMods so we send generation subresource only when user requested version parsing
yovadia@google.com [Mon, 10 Dec 2012 18:14:58 +0000 (18:14 +0000)]
Mods so we send generation subresource only when user requested version parsing
(-v option for cp and cat commands).

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@285 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded support for specifying src args for gsutil cp on stdin.
mfschwartz@google.com [Sun, 9 Dec 2012 15:41:27 +0000 (15:41 +0000)]
Added support for specifying src args for gsutil cp on stdin.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@284 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed gsutil test cp, which assumed it was run from gsutil install dir
mfschwartz@google.com [Thu, 6 Dec 2012 19:30:38 +0000 (19:30 +0000)]
Fixed gsutil test cp, which assumed it was run from gsutil install dir

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@283 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded the ability for the cp command to continue even if there is an error.
rein@google.com [Wed, 5 Dec 2012 00:23:29 +0000 (00:23 +0000)]
Added the ability for the cp command to continue even if there is an error.
This can be activated with the -c flag.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@282 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoUpdated docs about using setmeta with versioning enabled
mfschwartz@google.com [Mon, 3 Dec 2012 17:12:25 +0000 (17:12 +0000)]
Updated docs about using setmeta with versioning enabled

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@281 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agov3.20 release.
rein@google.com [Sat, 1 Dec 2012 00:33:34 +0000 (00:33 +0000)]
v3.20 release.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@280 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoImproved versioning support for remove command.
yovadia@google.com [Fri, 30 Nov 2012 23:17:07 +0000 (23:17 +0000)]
Improved versioning support for remove command.
* Explicitly disallow -a and -v options together.
* More robust deletes for current versions in "rm -a".
* General style improvements.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@279 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoImproved test runner.
jterrace@google.com [Fri, 30 Nov 2012 18:01:24 +0000 (18:01 +0000)]
Improved test runner.

* When running gsutil test, test results from individual test suites are
gathered and a summary is displayed at the end.

* Added return code handling to gsutil commands. Each RunCommand function can
now return an integer error code. The test command was updated to return a
non-zero error code when any of the tests don't pass.

* Changed test commands to run "set -o pipefail" before executing commands when
the bash shell is available to avoid missing possible errors when running tests.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@278 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded noclobber (-n) support to the cp command.
rein@google.com [Thu, 29 Nov 2012 01:01:41 +0000 (01:01 +0000)]
Added noclobber (-n) support to the cp command.

We check to see if the object/file exists before we initiate a copy. In
addition, if we're copying to gs we set the x-goog-if-generation-match to zero
so that we ensure that we don't clobber an existing object.

Usage of this command:
gsutil cp -n src dst

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@277 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed off-by-one error when reporting bytes transferred
jterrace@google.com [Wed, 28 Nov 2012 23:44:44 +0000 (23:44 +0000)]
Fixed off-by-one error when reporting bytes transferred

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@276 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agov3.19 release.
mfschwartz@google.com [Tue, 27 Nov 2012 02:38:04 +0000 (02:38 +0000)]
v3.19 release.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@275 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoImplement support for durable reduced availability storage.
pstemari@google.com [Tue, 27 Nov 2012 01:23:12 +0000 (01:23 +0000)]
Implement support for durable reduced availability storage.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@274 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdd support for object versions.
yovadia@google.com [Mon, 26 Nov 2012 23:23:39 +0000 (23:23 +0000)]
Add support for object versions.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@273 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoMade the tests more robust by running them through minidom.parseString
rein@google.com [Tue, 20 Nov 2012 01:31:58 +0000 (01:31 +0000)]
Made the tests more robust by running them through minidom.parseString

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@272 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFix expected output for setwebcfg tests.
yovadia@google.com [Sat, 17 Nov 2012 01:16:57 +0000 (01:16 +0000)]
Fix expected output for setwebcfg tests.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@271 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoBumping the version of the source code to 3.18 to match the binary.
rein@google.com [Fri, 16 Nov 2012 23:44:28 +0000 (23:44 +0000)]
Bumping the version of the source code to 3.18 to match the binary.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@270 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years ago1. Uploads would not resume in -q mode.
rein@google.com [Fri, 9 Nov 2012 22:38:47 +0000 (22:38 +0000)]
1. Uploads would not resume in -q mode.
2. setwebcfg and setcors tests were failing when comparing the resulting XML. The XML returned
from the server was formatted differently (no newlines)

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@269 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoIncorporated vapier@ mods to make version command not fail if CHECKSUM file missing.
mfschwartz@google.com [Sun, 4 Nov 2012 10:35:46 +0000 (10:35 +0000)]
Incorporated vapier@ mods to make version command not fail if CHECKSUM file missing.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@268 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoUpdated gsutil help dev instructions for how to check out source
mfschwartz@google.com [Fri, 2 Nov 2012 20:11:14 +0000 (20:11 +0000)]
Updated gsutil help dev instructions for how to check out source

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@267 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRemoved svn propset that had linked to boto via github gateway, because that
mfschwartz@google.com [Fri, 2 Nov 2012 19:18:42 +0000 (19:18 +0000)]
Removed svn propset that had linked to boto via github gateway, because that
gateway has become flaky.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@266 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRefactor gsutil such that most functionality exists in boto.
yovadia@google.com [Tue, 9 Oct 2012 17:15:36 +0000 (17:15 +0000)]
Refactor gsutil such that most functionality exists in boto.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@264 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoUpdated help text for "gsutil help setmeta" and "gsutil help metadata",
mfschwartz@google.com [Sat, 25 Aug 2012 23:28:32 +0000 (23:28 +0000)]
Updated help text for "gsutil help setmeta" and "gsutil help metadata",
to clarify and deepen parts of the documentation.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@263 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed one more place that outputted progress info when gsutil cp -q specified (during...
mfschwartz@google.com [Sat, 25 Aug 2012 00:10:11 +0000 (00:10 +0000)]
Fixed one more place that outputted progress info when gsutil cp -q specified (during streaming uploads)

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@262 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRelease 3.17.
mfschwartz@google.com [Fri, 17 Aug 2012 14:25:40 +0000 (14:25 +0000)]
Release 3.17.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@261 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoImplemented simplified syntax for setmeta command. The old syntax still works but...
mfschwartz@google.com [Fri, 17 Aug 2012 14:23:20 +0000 (14:23 +0000)]
Implemented simplified syntax for setmeta command. The old syntax still works but is now deprecated.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@260 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoClarified a detail in temp file change doc added in previous commit.
mfschwartz@google.com [Thu, 16 Aug 2012 16:28:20 +0000 (16:28 +0000)]
Clarified a detail in temp file change doc added in previous commit.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@259 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded help to gsutil cp -z option, to describe how to change where temp files are...
mfschwartz@google.com [Thu, 16 Aug 2012 16:26:00 +0000 (16:26 +0000)]
Added help to gsutil cp -z option, to describe how to change where temp files are written.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@258 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed race condition when multiple threads attempt to get an OAuth2 refresh token...
mfschwartz@google.com [Tue, 14 Aug 2012 22:41:29 +0000 (22:41 +0000)]
Fixed race condition when multiple threads attempt to get an OAuth2 refresh token concurrently.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@257 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRelease 3.16
mfschwartz@google.com [Tue, 14 Aug 2012 01:26:48 +0000 (01:26 +0000)]
Release 3.16

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@256 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoRelease 3.15.
mfschwartz@google.com [Sun, 12 Aug 2012 23:33:05 +0000 (23:33 +0000)]
Release 3.15.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@255 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed copy/paste error in config file documentation for https_validate_certificates...
mfschwartz@google.com [Sun, 12 Aug 2012 19:18:10 +0000 (19:18 +0000)]
Fixed copy/paste error in config file documentation for https_validate_certificates option.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@254 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoChanged gsutil to unset http_proxy environment variable if it's set, because it confu...
mfschwartz@google.com [Sun, 12 Aug 2012 18:56:08 +0000 (18:56 +0000)]
Changed gsutil to unset http_proxy environment variable if it's set, because it confuses boto. (Proxies should instead be configured via the boto config file.)

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@253 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoMade gsutil understand bucket subdir conventions used by various tools (like
mfschwartz@google.com [Sun, 12 Aug 2012 18:35:39 +0000 (18:35 +0000)]
Made gsutil understand bucket subdir conventions used by various tools (like
GCS Manager and CloudBerry), so if you cp or mv to a subdir you created with
one of those tools it will work as expected.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@252 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed naming bug when performing gsutil mv from a bucket subdir to and existing bucke...
mfschwartz@google.com [Sat, 11 Aug 2012 22:01:52 +0000 (22:01 +0000)]
Fixed naming bug when performing gsutil mv from a bucket subdir to and existing bucket subdir.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@251 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoHotfix to fix bug that stopped cross-provider copies into gs:// from working.
mfschwartz@google.com [Fri, 10 Aug 2012 20:45:28 +0000 (20:45 +0000)]
Hotfix to fix bug that stopped cross-provider copies into gs:// from working.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@250 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoImplemented gsutil setmeta command
mfschwartz@google.com [Sun, 5 Aug 2012 12:37:41 +0000 (12:37 +0000)]
Implemented gsutil setmeta command

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@249 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoMinor typo fixes.
mfschwartz@google.com [Fri, 3 Aug 2012 00:33:39 +0000 (00:33 +0000)]
Minor typo fixes.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@248 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoFixed performance bug when downloading a large object with Content-Encoding:gzip...
mfschwartz@google.com [Thu, 2 Aug 2012 12:40:02 +0000 (12:40 +0000)]
Fixed performance bug when downloading a large object with Content-Encoding:gzip, where decompression attempted to load entire object in memory. Also added "Uncompressing" log output if file is larger than 50M, to make it clear the download hasn't stalled.

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@247 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa

2 years agoAdded change needed to make resumable transfer progress messages not print when runni...
mfschwartz@google.com [Tue, 31 Jul 2012 20:50:20 +0000 (20:50 +0000)]
Added change needed to make resumable transfer progress messages not print when running gsutil cp -q

git-svn-id: svn://svn.chromium.org/gsutil/trunk/src@246 56d8b958-6b11-9cbd-d51d-67dd3ef4e9fa