chromiumos/chromite.git
11 months agoAssorted fixes for gerrit-on-borg support. 16/66216/14 master
ChromeBot [Mon, 29 Jul 2013 23:45:44 +0000 (16:45 -0700)]
Assorted fixes for gerrit-on-borg support.

This is a squash of commits by davidjames, vadimsh, and szager based
on live testing on a buildbot master.  A few notes:

- To roll this back, it should be sufficient to toggle the value of
USE_GOB in buildbot/constants.py.

- unit tests pass with USE_GOB=True/False, but a number of tests are
skipped when USE_GOB=True.  Those tests rely on statically-defined
expected results and/or 'magic constants' that are derived from the
known contents of the existing review DB.  They are obsolete when
testing gerrit-on-borg, and will need to be reimplemented to provide
equivalent coverage.

One-line commit notes from the squashed commits:

Fix QueryMultipleCurrentPatchset for case when len(changes) > 1.
Fetch commit message.
Make patch URL point to Gerrit HTML page rather than REST endpoint.
Fix invocation of gob_util.ResetReviewLabels
Handle no email address associated with gerrit account.
Fix o_params parameter
Pass un-munged change-id to ResetReviewLabels
Fix python string formatting char
Choose correct Gerrit helper in ValidationPool.ReloadChanges.
Use Gerrit REST API when submitting changes via Gerrit on Borg.
Coerce code review label values to strings.
Fix typo in ResetReviewLabels.
Fix dryrun behavior when posting reviews.
Update CQ to submit CLs even when debug mode.
Make QuerySingleRecord return GerritPatch when 'raw' set to False.
It used to return dict from Gerrit's JSON response.
Also ignore missing usernames in account dicts.
Re-enable tests.
Populate patch_dict["dependsOn"]
Allow GerritDependencies() to return sha1's.
When querying dependent changes, filter for the same project.
Drop port number from host when checking netrc entries.
Disable testIsAlreadyMerged when USE_GOB == True.
Test fixes, and re-enable dry-run behavior in gerrit.py

BUG=chromium:219617
TEST=buildbot/run_tests

Change-Id: I31f205010afab7a486cb506574887db872ed82d3
Reviewed-on: https://gerrit.chromium.org/gerrit/66216
Commit-Queue: Stefan Zager <szager@google.com>
Reviewed-by: Stefan Zager <szager@google.com>
Tested-by: Stefan Zager <szager@google.com>
11 months agoAdd CQ HWtests for Parrot. 41/66941/2
Scott Zawalski [Mon, 26 Aug 2013 15:51:27 +0000 (11:51 -0400)]
Add CQ HWtests for Parrot.

Average runtime for hwtests: 20 minutes.

TEST=out of band CQ running
BUG=chromium:196025

Change-Id: I4da50c36c767d1c5fc5d1ec7bffff01ee843d24b
Reviewed-on: https://gerrit.chromium.org/gerrit/66941
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>

11 months agochromiumos-build-stats cut off long string properties at 500 characters. 81/66881/2
Matt Tennant [Fri, 23 Aug 2013 22:19:59 +0000 (15:19 -0700)]
chromiumos-build-stats cut off long string properties at 500 characters.

According to error messages on the admin page of chromiumos-build-stats
appengine page, the cmd_line string property is sometimes greater than
500 characters now (in chrome-sdk stage of certain builders).  This is
apparently not allowed for string properties.

This change cuts strings greater than 500 characters down to 500
characters.

BUG=chromium:276377
TEST=pylint
TEST=Deployed on chromiumos-build-stats.appspot.com, then watched build
at
http://chromegw.corp.google.com/i/chromeos/builders/parrot%20canary/builds/1813
go through without errors in the chrome-sdk stage.

Change-Id: Id76e80c2bd3a4f48f92ea05fbf25c78d3e3361a4
Reviewed-on: https://gerrit.chromium.org/gerrit/66881
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Matt Tennant <mtennant@chromium.org>

11 months agocros_generate_breakpad_symbols: rewrite in python 34/65534/5
Mike Frysinger [Mon, 12 Aug 2013 00:08:19 +0000 (20:08 -0400)]
cros_generate_breakpad_symbols: rewrite in python

This will allow us to be able to better maintain/integrate/extend this
code rather than trying to use the existing bash version.

BUG=chromium:271251
TEST=`cros_generate_breakpad_symbols --board daisy` generates all the symbols
TEST=`./cros_generate_breakpad_symbols_unittest.py` passes
TEST+`./upload_symbols_unittest.py` passes

Change-Id: I75042561f7abaa9d53fb573e8567bb1ef8bcfb69
Reviewed-on: https://gerrit.chromium.org/gerrit/65534
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
11 months agoDon't generate the delta sysroot anymore. 78/66978/2
David James [Mon, 26 Aug 2013 19:17:58 +0000 (12:17 -0700)]
Don't generate the delta sysroot anymore.

The delta sysroot takes >25 minutes to generate for some reason.
Disable it while this can be investigated.

BUG=chromium:278512
TEST=cbuildbot_config_unittest.py

Change-Id: Ia6938f76371d9cc95bac1aa4fa282e738e5b9049
Reviewed-on: https://gerrit.chromium.org/gerrit/66978
Reviewed-by: Peter Mayo <petermayo@chromium.org>
Tested-by: David James <davidjames@chromium.org>
11 months agocbuildbot: Fix 'generator' has no len() error. 40/66840/2
David James [Fri, 23 Aug 2013 17:23:31 +0000 (10:23 -0700)]
cbuildbot: Fix 'generator' has no len() error.

This function only returns a short list, so there is no need to use a
generator here. This saves the caller the effort of needing to convert
the results to a list, and fixes the bug where the CQ does not reject
changes properly.

BUG=chromium:278337
TEST=Run cbuildbot with CL that breaks the CQ.

Change-Id: I70a207cb42fd9c9b95b11e7714a9b478ae1a3f09
Reviewed-on: https://gerrit.chromium.org/gerrit/66840
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Matt Tennant <mtennant@chromium.org>
11 months agoRemove chrome.pak from chrome_util.py 55/66855/3
James Robinson [Fri, 23 Aug 2013 20:15:48 +0000 (13:15 -0700)]
Remove chrome.pak from chrome_util.py

BUG=chromium:52609

Change-Id: If13e59629c40d8ee505a62711aa6a5b4aa708f15
Reviewed-on: https://gerrit.chromium.org/gerrit/66855
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: St├ęphane Marchesin <marcheu@chromium.org>
Tested-by: St├ęphane Marchesin <marcheu@chromium.org>
11 months agocbuildbot: include CrOS manifest version in build log summary 44/65244/3
Mike Frysinger [Thu, 8 Aug 2013 21:51:38 +0000 (17:51 -0400)]
cbuildbot: include CrOS manifest version in build log summary

When trying to figure out what manifest version a specific build is using,
log it in the summary text so people don't have to manually drill down in
the ManifestVersionedSync stage text.

BUG=None
TEST=`./buildbot/cbuildbot_stages_unittest.py` passes
TEST=`./buildbot/run_tests` passes

Change-Id: I0b46f6f28ffa54a1351935cc9e762f0f79b0d893
Reviewed-on: https://gerrit.chromium.org/gerrit/65244
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
11 months agoupload_command_stats: Include problematic URL in error message. 93/66393/4
Matt Tennant [Tue, 20 Aug 2013 17:46:06 +0000 (10:46 -0700)]
upload_command_stats: Include problematic URL in error message.

When upload_command_status receives an HTTPError from urlib2.urlopen it
should use the geturl() method of that object to indicate what URL it
failed to open.

BUG=chromium:276377
TEST=pylint
TEST=unittest
TEST=`cbuildbot --remote -g "I6f3c233d022700e23d06ad9051279c144e2b4ff5"
pre-cq-group` passes

Change-Id: I6f3c233d022700e23d06ad9051279c144e2b4ff5
Reviewed-on: https://gerrit.chromium.org/gerrit/66393
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Matt Tennant <mtennant@chromium.org>

11 months agoRemove the error_ok flag from cros_build_lib. 19/66419/2
David James [Tue, 20 Aug 2013 20:40:42 +0000 (13:40 -0700)]
Remove the error_ok flag from cros_build_lib.

BUG=chromium:204351
TEST=Remote trybot runs.

Change-Id: I6efa7d60e5044e1cc6cbf5958792fd28c0e0dcac
Reviewed-on: https://gerrit.chromium.org/gerrit/66419
Reviewed-by: Marc MERLIN <merlin@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
11 months agocros_portage_upgrade: fix clean up of stray objects 03/66503/3
Mike Frysinger [Wed, 21 Aug 2013 05:38:00 +0000 (01:38 -0400)]
cros_portage_upgrade: fix clean up of stray objects

The current clean up logic assumes that it's only upgrading a package
that has been committed to the tree.  If it's not (because a previous
run failed or whatever), the git rm step fails due to git not knowing
anything about the file:

$ cros_portage_upgrade --upgrade --force --board daisy:stumpy:x86-alex hdparm
...
cleaning up /home/vapier/trunk/src/third_party/portage-stable/sys-apps/hdparm
fatal: pathspec 'sys-apps/hdparm/hdparm-9.39.ebuild' did not match any files
Keeping upstream cache at /tmp/portage.
cros_portage_upgrade: Unhandled exception:
Traceback (most recent call last):
...
chromite.lib.cros_build_lib.RunCommandError: return code: 128
Failed command "'git' 'rm' '-rf' u'sys-apps/hdparm/metadata.xml'
u'sys-apps/hdparm/hdparm-9.20.ebuild' u'sys-apps/hdparm/hdparm-9.39.ebuild'
u'sys-apps/hdparm/files'", cwd=/home/vapier/trunk/src/third_party/portage-stable,
extra env={'GIT_PAGER': 'cat'}

Let's use the --ignore-unmatch flag to fix that, and then manually run
unlink on all the files that still exist.

BUG=chromium:212189
TEST=created a dummy file in sys-apps/hdparm and ran upgrade; did not see traceback and dummy file was cleaned

Change-Id: Ib8efd596ad3cda4c453f703f1a02feacca5ee35b
Reviewed-on: https://gerrit.chromium.org/gerrit/66503
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>

11 months agoAdded PGO configs for parrot and daisy 06/63606/4
Luis Lozano [Mon, 29 Jul 2013 18:34:35 +0000 (11:34 -0700)]
Added PGO configs for parrot and daisy

BUG=chromium:263589
TEST=buildbot/cbuildbot_config_unittest.py, buildbot/run_tests

Change-Id: I6bcedc32eafd6b1ef8e6a0559295158452c53921
Reviewed-on: https://gerrit.chromium.org/gerrit/63606
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
11 months agoEscape rsync commands to handle IPv6 addresses. 91/66291/2
David James [Tue, 20 Aug 2013 01:19:53 +0000 (18:19 -0700)]
Escape rsync commands to handle IPv6 addresses.

BUG=chromium:276065
TEST=Unit tests.
TEST=Verify SSH command works with IPv4 and IPv6 addresses.
TEST=Deploy over both IPv4 and IPv6.
Patch by: Marco Fucci <mfucci@google.com>
Change-Id: I25ab1ae9cbd0fa74b26c14dca6d9792e2c3f408c
Reviewed-on: https://gerrit.chromium.org/gerrit/66291
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
11 months agoSkip ChromeSDK stage on Lucid builders. 92/66392/3
David James [Tue, 20 Aug 2013 17:44:20 +0000 (10:44 -0700)]
Skip ChromeSDK stage on Lucid builders.

The ChromeSDK stage is passing on all Precise builders and failing on
all Lucid builders. We are planning to upgrade all Lucid builders.
Until that point, disable the ChromeSDK stage on the Precise builders.

BUG=chromium:276311
TEST=Confirm that platform.dist()[-1] is 'precise' on precise builder
TEST=Confirm that platform.dist()[-1] is 'lucid' on lucid builder
TEST=Run unit tests with both the ChromeSDK stage skipped and not
     skipped.
Change-Id: Idbfd34594b9f651d74d7291353f2cce566869eb3
Reviewed-on: https://gerrit.chromium.org/gerrit/66392
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Queue: Peter Mayo <petermayo@chromium.org>
Tested-by: Peter Mayo <petermayo@chromium.org>
11 months agoosutils: add Touch unittests 29/66229/2
Mike Frysinger [Mon, 19 Aug 2013 21:25:42 +0000 (17:25 -0400)]
osutils: add Touch unittests

I was thinking about adding other functionality so I started some
unittests, but then I ended up going a different route.  These tests
are still useful though.

BUG=None
TEST=`./osutils_unittest.py` passes

Change-Id: I90c8939e3806a0b50cfb4e88604353574e876003
Reviewed-on: https://gerrit.chromium.org/gerrit/66229
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
11 months agoupload_symbols: support uploading specific sym files 33/65533/3
Mike Frysinger [Sat, 10 Aug 2013 19:17:09 +0000 (15:17 -0400)]
upload_symbols: support uploading specific sym files

This makes manual upload of specific symbol files easy.  We sometimes
need to do this to restore missing symbol files from an older build.

BUG=chromium:266064
TEST=`upload_symbols --yes --testing /build/daisy/usr/lib/debug/breakpad/blkid/*/*.sym` works

Change-Id: I299d921949e6f0d7a763f36c0e8b6118371fcbc4
Reviewed-on: https://gerrit.chromium.org/gerrit/65533
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Marc MERLIN <merlin@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>

11 months agoRevert "Temporarily treat HWTest failures as if the lab were down." 06/66206/2
Chris Sosa [Mon, 19 Aug 2013 17:23:51 +0000 (10:23 -0700)]
Revert "Temporarily treat HWTest failures as if the lab were down."

Revert our temporary badness.

This reverts commit 8a5538ae1e3a9804110c95d5bfeb9380043c1406

BUG=chromium:273886
TEST=Clean revert.

Change-Id: I388ba7f07a9a90f5ecf6a1ddec167e0135babc2a
Reviewed-on: https://gerrit.chromium.org/gerrit/66206
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Chris Sosa <sosa@chromium.org>

11 months agoMake test type telemetry pass telemetry_unit. 00/66200/2
Scott Zawalski [Mon, 19 Aug 2013 15:38:33 +0000 (11:38 -0400)]
Make test type telemetry pass telemetry_unit.

There is no suite named telemetry, it is called telemetry_unit.

TEST=remote tryjob
BUG=chromium:264127

Change-Id: I3b55f6fdcf02c09d9741e1b8cfcb585ffebd4fa1
Reviewed-on: https://gerrit.chromium.org/gerrit/66200
Tested-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
11 months agocrosfw: Support building a image with both RO and RW components 71/62971/3
Simon Glass [Tue, 23 Jul 2013 04:35:01 +0000 (22:35 -0600)]
crosfw: Support building a image with both RO and RW components

The 'normal' U-Boot with command interpreter enabled and flashing support  is
too large to fit in IRAM whereas the RO U-Boot lacks these features. Add
support to crosfw for building an image where the RO U-Boot is in the RO
section and the RW U-Boot (built by the ebuild) is in the RW section.

BUG=chromium:244019
TEST=manual

$ emerge chromeos-u-boot
$ crosfw -V --ro  -a "--bootcmd vboot_twostop --force-rw" -z -F

See that it builds and image and flashes.

Then reset Pit and see that it runs through EFS, boots RW and then jumps to
the kernel.

Change-Id: I43dea7bc3b597836ce40ce693e55a85ab46736ed
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/62971

11 months agoFix telemetry suite name in constants for telemetry targets. 53/65953/2
Scott Zawalski [Thu, 15 Aug 2013 14:19:51 +0000 (10:19 -0400)]
Fix telemetry suite name in constants for telemetry targets.

TEST=remote tryjob
BUG=chromium:264127

Change-Id: I62920ccccdde06b87a009c8afdd392894d0cb7c9
Reviewed-on: https://gerrit.chromium.org/gerrit/65953
Tested-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>

11 months agocbuildbot: Add ChromeSDK stage. 44/63844/12
Ryan Cui [Wed, 7 Aug 2013 23:01:48 +0000 (16:01 -0700)]
cbuildbot: Add ChromeSDK stage.

Adds a ChromeSDK stage that:

1. Generates artifacts required for chrome-sdk for the board being
built.
2. Runs chrome-sdk against the newly generated artifacts, and builds
Chrome with it (using ninja and goma).
3. Runs deploy_chrome on the artifacts with --staging-only.

BUG=chromium:242998
TEST=Local, trybots

Change-Id: I6789fd1e842cbc72d0a72f9fdf1a8a90ea591a0b
Reviewed-on: https://gerrit.chromium.org/gerrit/63844
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
11 months agoDon't kick off an abort just because of a failed HWTest. 21/66021/2
David James [Wed, 14 Aug 2013 22:52:01 +0000 (15:52 -0700)]
Don't kick off an abort just because of a failed HWTest.

When compile failures occur, it's often early in the build process, so
it's a good idea to abort any hardware test processes that might be
running. On the other hand, once we've finally got to the hardware test
phase, HWTest finishes and fails, it doesn't make sense to abort the
other HWTests.

BUG=chromium:271874
TEST=Run an example run with failing HWTests and confirm no abort runs.

Change-Id: I66cc8df2c5bc55aeb2e00a93a18ef6d999d0d197
Reviewed-on: https://gerrit.chromium.org/gerrit/66021
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
11 months agocbuildbot_stages: add --notests support for ArchiveDeltaSysroot 29/65629/2
Thiago Goncales [Mon, 12 Aug 2013 18:01:47 +0000 (11:01 -0700)]
cbuildbot_stages: add --notests support for ArchiveDeltaSysroot

BUG=chromium:271218
TEST=trybot run with --notests.

Change-Id: I65dcefdfa403cc5d8de221bb11458c055dea900e
Signed-off-by: Thiago Goncales <thiagog@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/65629
Reviewed-by: Mike Frysinger <vapier@chromium.org>
11 months agoTemporarily treat HWTest failures as if the lab were down. 88/65988/2
Chris Sosa [Thu, 15 Aug 2013 21:08:57 +0000 (14:08 -0700)]
Temporarily treat HWTest failures as if the lab were down.

BUG=chromium:273886
TEST=Unittests + pylint.

Change-Id: I969fc49ec1cd2b545ef25e289c8de1a38f753121
Reviewed-on: https://gerrit.chromium.org/gerrit/65988
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
11 months agoAdd a configuration for building stumpy testbed-ap images 23/64823/4
Christopher Wiley [Tue, 6 Aug 2013 21:44:30 +0000 (14:44 -0700)]
Add a configuration for building stumpy testbed-ap images

TEST=None
BUG=chromium:269134

Change-Id: I59e560cea8f91fa85aa134d4e544f0242f29d8dd
Reviewed-on: https://gerrit.chromium.org/gerrit/64823
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Christopher Wiley <wiley@chromium.org>
Commit-Queue: Christopher Wiley <wiley@chromium.org>

11 months agoUpload qemu image with artifacts. 25/65425/3
David James [Wed, 14 Aug 2013 23:02:22 +0000 (16:02 -0700)]
Upload qemu image with artifacts.

BUG=chromium:264906
TEST=Run trybot and verify qemu image is uploaded now.

Change-Id: Ifffb3a54da61a5fb5ccaa9fbea913678d0ba5fba
Reviewed-on: https://gerrit.chromium.org/gerrit/65425
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
11 months agogenerate_delta_sysroot: add --skip-tests. 28/65628/2
Thiago Goncales [Sun, 11 Aug 2013 03:09:20 +0000 (20:09 -0700)]
generate_delta_sysroot: add --skip-tests.

The flag --skip-tests does not build autotest packages during the delta
sysroot creation. It is particularly useful for when the local sysroot
that is used as a binhost does not have said packages.

Also:
  Create unpackonly sysroot before full sysroot. Small optimization which will
    let us parallelize better, by doing least complex operation at the
    beginning, when most of the parallel tasks are happening in the Paladins.
  Ignore /var/cache/. The cache can be very different in the paladin
    compared to the dev machine.
  Ignore /packages/. Speeds up batch creation and batch read.

BUG=chromium:271218, chromium:271709
TEST=unittest

Change-Id: Ibfbbf30ddf636063b788342fc5affe87729de099
Signed-off-by: Thiago Goncales <thiagog@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/65628
Reviewed-by: Ryan Cui <rcui@chromium.org>
11 months agoReport what changes were in a given CQ run in metadata.json. 05/65805/5
David James [Wed, 14 Aug 2013 03:29:57 +0000 (20:29 -0700)]
Report what changes were in a given CQ run in metadata.json.

It is helpful to know what changes are included in a CQ run
for the purposes of gathering stats.

I have added data that looks like this:

"changes": [{"gerrit_number": "1234", "patch_number": "1", "internal": false}]

BUG=chromium:270774
TEST=New unit test. mario-paladin trybot run.

Change-Id: I97816afed03eb50d4bb2abeca3c779b9c0e90d17
Reviewed-on: https://gerrit.chromium.org/gerrit/65805
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
11 months agocbuildbot: Enable perf testing on PGO config. 26/65326/12
Ahmad Sharif [Fri, 9 Aug 2013 17:28:36 +0000 (13:28 -0400)]
cbuildbot: Enable perf testing on PGO config.

BUG=chromium:267837
TEST=remote trybot.

Change-Id: Idcaa27279f9360723534790a8b4b6dba8919ecc6
Reviewed-on: https://gerrit.chromium.org/gerrit/65326
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: asharif <asharif@chromium.org>
Commit-Queue: asharif <asharif@chromium.org>

11 months agoStart uploading hw test artifacts for parrot. Set builders as important. 97/65697/4
Scott Zawalski [Tue, 13 Aug 2013 12:22:20 +0000 (08:22 -0400)]
Start uploading hw test artifacts for parrot. Set builders as important.

Shuffle long running x86 unittest to parrot32-paladin
Builders updated to important:
parrot32
parrot64
wolf
stout32

TEST=unittests
BUG=chromium:196025

Change-Id: I42d4b3332ed7547a830353343ee8c9ea4edd0419
Reviewed-on: https://gerrit.chromium.org/gerrit/65697
Tested-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
11 months agocbuildbot_stages: add build and archive stage of delta sysroot. 94/57994/11
Thiago Goncales [Fri, 7 Jun 2013 22:51:38 +0000 (15:51 -0700)]
cbuildbot_stages: add build and archive stage of delta sysroot.

This implements an extra step in the Archive Stage, which creates a
delta sysroot tarball using generate_delta_sysroot.

This is enable by config create_delta_sysroot, enabled in paladin
builds.

BUG=chromium:263845
TEST=unittest; trybot run on link-paladin config.

Change-Id: I8f92c8c68a1e10d4e2c2e69de00d3d4b58a735d1
Signed-off-by: Thiago Goncales <thiagog@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/57994
Reviewed-by: David James <davidjames@chromium.org>
11 months agocommandline: sort log levels 32/65532/2 release-R30-4537.B
Mike Frysinger [Sun, 11 Aug 2013 23:02:10 +0000 (19:02 -0400)]
commandline: sort log levels

The sorting of this constant doesn't matter to the code, but it does
get used in the --help output.  So let's sort the var so it indicates
the relative levels to users.  Otherwise they need to look up the
logging documentation to figure out which is higher priority.

BUG=None
TEST=`upload_symbols --help` shows log levels sorted correctly

Change-Id: Idcb34472462b3726c1cdbf95198c151de662b1c6
Reviewed-on: https://gerrit.chromium.org/gerrit/65532
Reviewed-by: Marc MERLIN <merlin@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>

11 months agoAdd stumpy HWtests to the CQ. 85/65585/2
Scott Zawalski [Mon, 12 Aug 2013 14:10:20 +0000 (10:10 -0400)]
Add stumpy HWtests to the CQ.

Average turnaround time: 30 minutes.

TEST=Ran stumpy CQ for a week and observed no notifications.
BUG=chromium:196025

Change-Id: I10a5c57d7f5a67c2a7824ac129e18b49f7e322e3
Reviewed-on: https://gerrit.chromium.org/gerrit/65585
Tested-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
11 months agoBig code reorg, show multiple version numbers 93/64693/4
Marc MERLIN [Mon, 5 Aug 2013 22:12:45 +0000 (15:12 -0700)]
Big code reorg, show multiple version numbers

- big code reorg to fix the OOP issues in this code and remove globals.
- sort scanned licenses so that ordering doesn't depend on local filesystem
  readdir() ordering.
- Added version numbers to packages because they are necessary for some
  packages we have multiple of, and some with different licenses of
  course, or it'd be too easy :)
  - sys-devel/autoconf-2.13
  - sys-devel/autoconf-2.68
  - dev-libs/libusb-0.1.12-r7
  - dev-libs/libusb-1.0.9
  - dev-libs/libnl-1.1-r1
  - dev-libs/libnl-3.2.14
  Up to now, we failed to show the licenses for each separately.
- debug mode is not needed for normal operation anymore.
- Added proper argument parsing and ability to generate the license for
  a single package
- Hacked virtual packages that don't really exist are now fed to the
  licensing object instead of being hardcoded in the middle of the
  logic.
- The old code wasn't encoding aware, it outputed the same encoding that
  it found on disk. This caused some problems with python processing
  that tried to do unicode decoding on the fly, and failed on latin1
  text. This new code uses codecs do to explicit utf-8 load of files,
  and if that fails try again as a latin1 import, outputting utf-8.
- Package errors are now reported via exceptions.
- Special code to handle GPL-2+ and LGPL-2+ by actually displaying a
  real license.
- Now display list of stock licenses at the end of output.

BUG=chromium:197970

TEST=I diffed the HTML before and after this code change and the output is
similar baring some whitespace changes and 2 entries for packages we have more
than one version of. Now the diff is of course much bigger since I added package
version numbers, but I diffed it before that to ensure the big code reorg didn't
change the output unexpectedly.
The next test with the GPL-2+ patch added shows that this case is
handled properly now.

Change-Id: I0ad34d3c9e8aa6858781daae600be42c5c60be8a
Reviewed-on: https://gerrit.chromium.org/gerrit/64693
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Queue: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
11 months agoparallel_emerge: add --unpackonly. 56/62856/13
Thiago Goncales [Wed, 17 Jul 2013 17:26:35 +0000 (10:26 -0700)]
parallel_emerge: add --unpackonly.

The --unpackonly flag makes parallel-emerge fetch and then unpack all
packages, but not emerge them. This was accomplished by creating an
unpack queue, which, if --unpackonly is set, is filled as the fetch queue
is completed.

If we desire to work with the binary packages only of a project, the
workflow is to run --fetchonly and then extracting each package. By
adding --unpackonly, we can parallelize this with fetchonly and make the
entire workflow faster.

BUG=chromium:264240
TEST=Compared sysroot using --unpackonly with using --fetchonly and then
     unpacking all packages.

Change-Id: I3d9acb7032f036b786e3cba9843b7720471eecdb
Signed-off-by: Thiago Goncales <thiagog@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/62856
Reviewed-by: David James <davidjames@chromium.org>
11 months agoImprove checkout determination. 00/65100/6
Ryan Cui [Thu, 8 Aug 2013 18:41:57 +0000 (11:41 -0700)]
Improve checkout determination.

Instead of hardcoding the order of what checkout type we look for, find
the nearest checkout root, whether it be gclient, repo, or submodule.
This is needed to handle nested checkouts.

Previously we looked for the .gclient file  first in all the parent
directories, whereas now we look for a repo, gclient, and git checkout
at each level.

For example, in the previous code, a directory hierarchy of
  /a/.gclient
  /a/b/.repo
and with cwd=/a/b would give /a as the root since we were looking for a
gclient checkout first, but we really want to find the repo checkout at
/a/b. The current code finds the closest checkout root of any kind, so
it would return the right value of /a/b.

BUG=None
TEST=Locally, remote trybots.

Change-Id: I14f0010e64e5a1d18a119ac4bd9e0c2df776a373
Reviewed-on: https://gerrit.chromium.org/gerrit/65100
Commit-Queue: <rcui@google.com>
Reviewed-by: <rcui@google.com>
Tested-by: <rcui@google.com>
Reviewed-by: Ryan Cui <rcui@chromium.org>
11 months agoPrevent branch-util from being run on the remote trybots. 60/65260/3
Ryan Cui [Thu, 8 Aug 2013 23:29:51 +0000 (16:29 -0700)]
Prevent branch-util from being run on the remote trybots.

We don't want everyone to create branches, and we want to rely on
gerrit's ACL's to control branch creation.  So prevent --remote jobs for
branch-util.  This is not meant has a strong means of prevention, only
one layer of deterrent, useful for preventing users from accidentally
launching remote tryjobs.

BUG=chromium:260969
TEST=Locally

Change-Id: I880cde1dfa26b6fd9c574cf277b61e6598d1f880
Reviewed-on: https://gerrit.chromium.org/gerrit/65260
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Queue: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
11 months agoAdd generate_delta_sysroot.py 53/56653/19
Thiago Goncales [Thu, 8 Aug 2013 01:45:22 +0000 (18:45 -0700)]
Add generate_delta_sysroot.py

This script generates a tarball containing a sysroot that can be patched
over extracted prebuilt package contents to create a complete sysroot.

Usage:
  generate_delta_sysroot --board=<board> --out-dir=<out_dir>
  [--out-file=<out_file> --out-batch=<batch>]

  Creates a delta sysroot tarball of name <out_file> in <out_dir>
  containing batch file <batch> for board <board>

The script works by:
  - Emerges all packages for a board into a build sysroot directory.
  - Unpack all packages for a board into a temp sysroot directory.
  - Creates a batch file using:
    rsync -rplgoDc --delete --write-batch=<batch> <build_sys> <tmp_sys>
  - Puts the batch file inside a tarball.

This batch file can be applied to any directory identical to the
temporary folder. This can be achieved by using build_packages with an
--unpackonly flag. Then, in order to get a complete sysroot, it's sufficient
to use:

Note that the script assumes that there exists a built sysroot already.
In fact, the packages will be fetched from that sysroot.

rsync -rplgoDc --delete --read-batch=<batch> <extract_pkgs_dir>

BUG=None
TEST=scripts/generate_delta_sysroot_unittest.py

Change-Id: I22dafd8ba900cb48956a32ff87d3d1206f14d322
Signed-off-by: Thiago Goncales <thiagog@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/56653
Reviewed-by: Ryan Cui <rcui@chromium.org>
11 months agoFix syntax error in GoB query parameter. 34/65034/2
Stefan Zager [Wed, 7 Aug 2013 23:59:00 +0000 (16:59 -0700)]
Fix syntax error in GoB query parameter.

TBR=davidjames@chromium.org

Change-Id: Id812263f1a366ebe742284a143302f58ebd2bdcb

11 months agoFix bug in custom version key generation. 11/64711/2 stabilize-4512.B
Ryan Cui [Mon, 5 Aug 2013 23:41:35 +0000 (16:41 -0700)]
Fix bug in custom version key generation.

Instead of using '%' which is a shell special character, and may be
expanded in certain cases, use the safer '__' separator.  We use double
underscore for better readability.

BUG=None
TEST=Ran on a buildbot and locally.

Change-Id: If82f74effc3f977f5e58d543a097198688347e21
Reviewed-on: https://gerrit.chromium.org/gerrit/64711
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
11 months agochrome-sdk: exit with the exit code of the command being run. 99/64799/4
Ryan Cui [Tue, 6 Aug 2013 18:50:59 +0000 (11:50 -0700)]
chrome-sdk: exit with the exit code of the command being run.

An error code from a command being executed in the chrome SDK
environment should be passed through by the SDK.  So have the SDK exit
with the exit code of the command it is running.

BUG=None
TEST=Local

Change-Id: I31a100e06c775b22fc151fbb283b57e579819dd3
Reviewed-on: https://gerrit.chromium.org/gerrit/64799
Commit-Queue: Ryan Cui <rcui@chromium.org>
Reviewed-by: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
11 months agochrome-sdk: Add --sdk-path support 87/63687/8
Ryan Cui [Mon, 29 Jul 2013 23:05:37 +0000 (16:05 -0700)]
chrome-sdk: Add --sdk-path support

You can specify either a local directory or a gs:// path to pull SDK
components from with --sdk-path.  This is used by the builders to run
chrome-sdk with artifacts that have not yet been uploaded.

BUG=chromium:265607
TEST=Remote trybots

Change-Id: Ib743eb1bdbd0b5c53d5ed02aab01c1f349c40de9
Reviewed-on: https://gerrit.chromium.org/gerrit/63687
Commit-Queue: Ryan Cui <rcui@chromium.org>
Reviewed-by: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
11 months agochrome-sdk: add --cwd flag. 43/63843/5
Ryan Cui [Wed, 31 Jul 2013 00:30:53 +0000 (17:30 -0700)]
chrome-sdk: add --cwd flag.

The --cwd flag controls which directory the SDK shell starts the user
off at.

Also fixes a bug with --chrome-src not being passed through properly.

BUG=chromium:265607
TEST=Locally

Change-Id: I2174b17aaa113a315f5eb111a998657a6545ce14
Reviewed-on: https://gerrit.chromium.org/gerrit/63843
Commit-Queue: Ryan Cui <rcui@chromium.org>
Reviewed-by: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
11 months agoMove metadata upload functionality into ArchivingStage 68/63268/6
Ryan Cui [Wed, 24 Jul 2013 22:51:26 +0000 (15:51 -0700)]
Move metadata upload functionality into ArchivingStage

So that all stages derived from ArchivingStage can upload metadata
information for itself.

Stages that are not the Report stage will upload a metadata_<stage>.json
file instead of the final metadata.json file.  This is to prevent
overwriting of existing metadata files.

BUG=chromium:253090
TEST=local, remote trybots.

Change-Id: Ibac614d99370645cc55075433a6f994dc375cd3e
Reviewed-on: https://gerrit.chromium.org/gerrit/63268
Commit-Queue: Ryan Cui <rcui@chromium.org>
Reviewed-by: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
11 months agobuildbot: pgo: cleanup both pgo_generate and pgo_use images. 53/62853/7
Ahmad Sharif [Mon, 22 Jul 2013 17:51:52 +0000 (13:51 -0400)]
buildbot: pgo: cleanup both pgo_generate and pgo_use images.

Currently the buildbot cleans up only the first config in a group of
configurations. The PGO builder runs out of disk because of this. So fix
buildbot to clean up all the configs in a group of configs.

BUG=chromium:256240
TEST=cbuildbot ...

Change-Id: I2ef3c313234406fe14e78b07f9bd5e899104f2ff
Reviewed-on: https://gerrit.chromium.org/gerrit/62853
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Peter Mayo <petermayo@chromium.org>
Commit-Queue: asharif <asharif@chromium.org>
Tested-by: asharif <asharif@chromium.org>
11 months agoCheck for a repo checkout last. 67/63267/4
Ryan Cui [Wed, 24 Jul 2013 01:13:28 +0000 (18:13 -0700)]
Check for a repo checkout last.

When determining the type of checkout we're in, do the 'repo' checkout
test last, since in some cases a Chrome checkout may be embedded inside
a repo checkout, and we want to detect the Chrome checkout rather than
the repo checkout.

BUG=None
TEST=Manual

Change-Id: I2aad1d62c22d411a79ad9324cdcaeb919675734b
Reviewed-on: https://gerrit.chromium.org/gerrit/63267
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
11 months agochrome-sdk: Export cached gsutil path. 41/64341/2
Ryan Cui [Fri, 2 Aug 2013 18:10:51 +0000 (11:10 -0700)]
chrome-sdk: Export cached gsutil path.

To support running devserver from within the SDK shell.

BUG=None
TEST=Local

Change-Id: I214598c4c4f8125ecf5109823da8136c005a3715
Reviewed-on: https://gerrit.chromium.org/gerrit/64341
Tested-by: Ryan Cui <rcui@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Ryan Cui <rcui@chromium.org>

11 months agoUse binary packages in cros_generate_sysroot. 52/63152/4
David James [Wed, 31 Jul 2013 22:34:45 +0000 (15:34 -0700)]
Use binary packages in cros_generate_sysroot.

cros_generate_sysroot currently rewrites the sysroot itself, breaking
any scripts that need to build images from the sysroot. This is
aggravated by the fact that cros_generate_sysroot builds with the wrong
use flags. We have fixed this problem by updating updating
cros_generate_sysroot to pass in the right use flags and to use
--usepkgonly.

BUG=chromium:218085
TEST=All unit tests, all release trybots.
TEST=Compile built tarball with cros chrome-sdk.

Change-Id: I6661e5ba6b0b18cd6292d0ac48fd428a232fadbf
Reviewed-on: https://gerrit.chromium.org/gerrit/63152
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>

11 months agocrosfw: Create eMMC magic flasher 98/63998/5
Michael Pratt [Wed, 31 Jul 2013 23:27:29 +0000 (16:27 -0700)]
crosfw: Create eMMC magic flasher

The '-M' option allows building a magic flasher to flash the eMMC boot
partition.

BUG=chromium:254233
TEST='crosfw -b daisy_spring -VSF' flashes to SPI flash
'crosfw -b daisy_spring -VSM' flashes to eMMC

Change-Id: Idff1ff6ab627870c13e039cc26680468ed14f416
Signed-off-by: Michael Pratt <mpratt@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/63998
Reviewed-by: Vincent Palatin <vpalatin@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
12 months agoReplace nowithdebug with use flags. 31/62931/2
David James [Mon, 22 Jul 2013 23:18:37 +0000 (16:18 -0700)]
Replace nowithdebug with use flags.

The nowithdebug option was added back when cbuildbot did not support use
flags. Now that cbuildbot supports use flags, we no longer need the
nowithdebug option.

This CL has the side effect of making the -cros-debug flag get passed
in more places, including in build_image. This happens to be necessary
to upgrade to Portage 2.1.11.50, because this new version of Portage
is more strict about checking use flags.

BUG=chromium:218085
TEST=Release trybot runs with and without new version of Portage.

Change-Id: If281135fa75d63e1ede83dcd8803efaa4c0ad623
Reviewed-on: https://gerrit.chromium.org/gerrit/62931
Reviewed-by: Marc MERLIN <merlin@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>

12 months agoEnsure TimeoutError exceptions actually fail the build. 84/63484/3
David James [Fri, 26 Jul 2013 19:06:22 +0000 (12:06 -0700)]
Ensure TimeoutError exceptions actually fail the build.

Currently, a TimeoutError doesn't actually fail the build, because of a
misunderstanding of how exception handling works in cbuildbot_stages.py.
Nobody actually checks the value of PerformStage, so returning a value
there doesn't cause the stage to fail. Update HWTest to use the correct
method of handling TimeoutError exceptions so that we actually fail the
build, and add a unit test.

BUG=chromium:263677
TEST=New unit test, trybot run w/TimeoutError

Change-Id: I7e1f96b29c6758d111b421a6c76f61e244e2d16d
Reviewed-on: https://gerrit.chromium.org/gerrit/63484
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>

12 months agogdata_lib: Retry on 500 errors, and do not assume chromium.org domain. 30/64030/2
Matt Tennant [Thu, 1 Aug 2013 03:06:33 +0000 (20:06 -0700)]
gdata_lib: Retry on 500 errors, and do not assume chromium.org domain.

This extends the automatic retry logic in gdata_lib to include retries
on 500 errors.  It also accepts email addresses with domains other than
chromium.org for connecting to Google Drive.

BUG=None
TEST=pylint
TEST=Used with local script for uploading contents of crostools repo to
a Drive spreadsheet.

Change-Id: Ica301a224148efc2bd8ce77ce07adf6447bc5d41
Reviewed-on: https://gerrit.chromium.org/gerrit/64030
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Commit-Queue: Matt Tennant <mtennant@chromium.org>

12 months agoAdd hwtest support to the falco release builder 23/63923/4
Fang Deng [Wed, 31 Jul 2013 17:42:33 +0000 (10:42 -0700)]
Add hwtest support to the falco release builder

Make falco-release builder run with hwtest.

BUG=chromium:266329
TEST=None

Change-Id: I7e35a4cb92674ec6ea5f21cd415c670a0f3c9115
Reviewed-on: https://gerrit.chromium.org/gerrit/63923
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
12 months agoRemove boards restrictions for cbuildbot --hwtest 05/63805/7
Dan Shi [Tue, 30 Jul 2013 23:22:12 +0000 (16:22 -0700)]
Remove boards restrictions for cbuildbot --hwtest

Instead of fail the build, post a warning that hwtest step may fail if the
specified board is not a released platform or it is a generic overlay.

BUG=chromium:264871
TEST=tested with cbuildbot --remote, pass unittest in chroot:
     ./cbuildbot_config_unittest.py

Change-Id: Ibbf597a23d2f171c81e864377dc38cc24e449d3c
Reviewed-on: https://gerrit.chromium.org/gerrit/63805
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>

12 months agoSignificantly improved the output (700KB of missing licenses). 89/51489/8
Marc MERLIN [Thu, 16 May 2013 19:17:21 +0000 (12:17 -0700)]
Significantly improved the output (700KB of missing licenses).

- This new version is much more bulletproof, it now checks for errors
  and dies on them.
- The biggest change is that it also now relies on gentoo provided
   licenses in ebuild vs some incomplete and often wrong scanning of source
  code as the primary method. This fixes many licenses we were missing
  (700KB's worth).
- If BSD or MIT is found in the licenses, then it scans the source code
  to find all copyright/licenses found to comply with copyright
  attribution.
- Simple but working logic to deal with license OR vs AND was added to
  know whether we even have to hunt for a BSD license in a dual OR license
  scenario. Only in the AND scenario is the BSD attribution looked
  for.
- I had to add a fontconfig license that was missing.
- I also added an MIT license for Mesa (hidden in license.html which we
  don't parse).
- Added media-libs/harfbuzz-0.9.18-r2: Old-MIT is now also pointed to
  a license that needs attribution.
- Added GoogleXXXInput packages that are proprietary to the exclude list.
- There is now a list of license tokens we should ignore, which in turn
  allows to error on any license that can't be read from disk (they were
  silently ignored before).
- Removed hackish escape logic inside ReadStockLicense and
  ExtractLicenses: do the right thing and don't call them if not needed.
- Added a basic sanity check when reading ebuilds, die if LICENSES is
  not defined there.
- After manually reviewing license diffs, noticed that xz-utils has an
  incomplete LICENSE record in the ebuild, added a fix for it.
- After reviewing license diffs, I noticed that for a few licenses like
  as-is, bzip2, of, ncsa, and psf, it's actually better to include the
  license file from the code instead of the gentoo template.
- As a result, added LOOK_IN_SOURCE_LICENSES to list licenses that we
  can complement with a copy of what's in the source if any.
- Reorder licenses to show scanned ones first since they're usually
  better.

BUG=chromium:197970

TEST=I had a look at the diff output from before and after this change.
I can definitely see that a lot of missing licenses were added.
Only one license was removed:
glic has the GPL scanned twice (glib + pkg-config) and is now only
shown once.
Everything else in the diff are licenses that were added or modified a
bit from the included copy to the stock copy.
Things to note:
- GoogleCheineInput-* is removed since license is proprietary
- during testing the new harfbuzz included Old-MIT which was added to
  the list of https://b.corp.google.com/issue?id=5028287

Change-Id: I86b52574a897d188ec2226167aee382cdf5d38dc
Reviewed-on: https://gerrit.chromium.org/gerrit/51489
Reviewed-by: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
Commit-Queue: Marc MERLIN <merlin@chromium.org>

12 months agoUse normlized ref names during branching/manifest fixup. 97/63397/3
Ryan Cui [Thu, 25 Jul 2013 20:39:08 +0000 (13:39 -0700)]
Use normlized ref names during branching/manifest fixup.

Use long form refs/heads/* names for the branch during branching rather
than the short form name, because:

1. This is the standard for manifests.
2. Avoids ambiguity when running git commands.

BUG=None
TEST=cbuildbot --local --debug branch-util ...

Change-Id: I37a33d15f5b4679e730fd6bc732a04e8bd2d2925
Reviewed-on: https://gerrit.chromium.org/gerrit/63397
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
Commit-Queue: Ryan Cui <rcui@chromium.org>

12 months agoAdd PNaCl files from chrome into chromeos. 81/63481/2
Jan Voung [Fri, 26 Jul 2013 18:40:06 +0000 (11:40 -0700)]
Add PNaCl files from chrome into chromeos.

BUG=chromium:221381
TEST=none (trybots)

Change-Id: I05e30954aa650fe84655ba153e355fe34abc7a84
Reviewed-on: https://gerrit.chromium.org/gerrit/63481
Reviewed-by: Dmitry Polukhin <dpolukhin@chromium.org>
Commit-Queue: Jan Voung <jvoung@chromium.org>
Tested-by: Jan Voung <jvoung@chromium.org>
12 months agoFixed pach failed to apply message. 02/63302/3
Marc MERLIN [Thu, 25 Jul 2013 00:42:31 +0000 (17:42 -0700)]
Fixed pach failed to apply message.

BUG=chromium:216488

TEST=cros lint

Change-Id: Id6a7a3da98bbffa81ce67eb2b22e5f0a9e294862
Reviewed-on: https://gerrit.chromium.org/gerrit/63302
Commit-Queue: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
Reviewed-by: Marc MERLIN <merlin@chromium.org>
12 months agoautotest_quickmerge: Pick up the default board if one is available. 27/61227/4
Aviv Keshet [Thu, 18 Jul 2013 17:52:04 +0000 (10:52 -0700)]
autotest_quickmerge: Pick up the default board if one is available.

autotest_quickmerge requires the --board flag for the current board. After this
CL, it will pick up the default board from src/scripts/.default_board if one is
available.
This brings the behaviour of autotest_quickmerge in line with other scripts we
use.

BUG=chromium:258238
TEST=Manually tested that autotest_quickmerge picks up the default board. Also
tested that it skips ill-formed default board names.

Change-Id: If78fa3c8f0130dcc6adf7b63dd7d89859dabd3c9
Reviewed-on: https://gerrit.chromium.org/gerrit/61227
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
12 months agoDisable VM Tests on parrot64 where devserver tests are flaky for now. 76/63376/3
Chris Sosa [Thu, 25 Jul 2013 19:37:23 +0000 (12:37 -0700)]
Disable VM Tests on parrot64 where devserver tests are flaky for now.

BUG=chromium:251309
TEST=unittest

Change-Id: I3c0ce9e8f388ea82297fa5d6255978a3479b4604
Reviewed-on: https://gerrit.chromium.org/gerrit/63376
Commit-Queue: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
12 months agoFile bugs on Canary HWTests 74/63374/2
beeps [Fri, 21 Jun 2013 01:15:34 +0000 (18:15 -0700)]
File bugs on Canary HWTests

Since we don't want to file bugs on paladin hwtests
but do want to file bugs on canary hwtests, we need
to pipe the flag from chromite instead of setting it
in the suite control file.

TEST=Deduped bugs on chromium via a cbf dummy suite.
BUG=chromium:212058

Change-Id: Idb8d15025765043f817f7cfa1a1002d8cb54c087
Reviewed-on: https://gerrit.chromium.org/gerrit/63374
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>

12 months agoHave manifest parsing code handle short-form branch names 84/63384/2
Ryan Cui [Thu, 25 Jul 2013 21:07:23 +0000 (14:07 -0700)]
Have manifest parsing code handle short-form branch names

Robustifies the code, by adding 'refs/heads/' to short form manifest
revisions.  i.e., "revision='master'" in the manifest will translate to
a normalized revision of 'refs/heads/master'.

Prevents failures during 'git checkout', where the short-form branch
name is ambiguous.

BUG=None
TEST=Local, trybot.

Change-Id: I3b0eded405661cac24219e9d56bffe7581ac8604
Reviewed-on: https://gerrit.chromium.org/gerrit/63384
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
12 months agoRevert "Adding file_bugs to stout canary run_suite invocation." 54/63354/2
Prashanth Balasubramanian [Thu, 25 Jul 2013 17:17:03 +0000 (10:17 -0700)]
Revert "Adding file_bugs to stout canary run_suite invocation."

The project hosting backend is rejecting the credentials we've used to file bugs thus far.
Reverting till the issue has been resolved.
This reverts commit 07ab9e3af96beb5b641d8bf617e3073c5b07e34d

Change-Id: I984663be7d4c5d7218a7268d4f9afba16836665c
Reviewed-on: https://gerrit.chromium.org/gerrit/63354
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
12 months agoRevert "File bugs on Canary HWTests" 52/63352/2
Prashanth Balasubramanian [Thu, 25 Jul 2013 17:15:27 +0000 (10:15 -0700)]
Revert "File bugs on Canary HWTests"

The project hosting backend is rejecting the credentials we've used to file bugs thus far, reverting till the issue has been resolved.

This reverts commit 51f0b25437e356a6a881b546947b782aab67ffee

Change-Id: I354afa1a5832b4bcde5ba06aa01bae4e97682b63
Reviewed-on: https://gerrit.chromium.org/gerrit/63352
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
12 months agoAdd telemetry test configurations. 63/62763/4
Mike Stipicevic [Sat, 20 Jul 2013 00:43:20 +0000 (17:43 -0700)]
Add telemetry test configurations.

CQ-DEPEND=CL:62766
BUG=chromium:234843
TEST=buildbot/cbuildbot_config_unittest.py
Change-Id: I12001517d80b33a156bfa102f0fadb6ad708d7b0
Reviewed-on: https://gerrit.chromium.org/gerrit/62763
Tested-by: Mike Stipicevic <xusydoc@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Mike Stipicevic <xusydoc@chromium.org>

12 months agoSet nacl_helper as executable. It is now being used, and NaCl needs it +x 24/63224/2
Jan Voung [Wed, 24 Jul 2013 18:39:52 +0000 (11:39 -0700)]
Set nacl_helper as executable.  It is now being used, and NaCl needs it +x

A chrome change:
https://src.chromium.org/viewvc/chrome?view=rev&revision=212997
Made nacl_helper be used.  This works on linux, but not chromeos.

BUG=chromium:263525
TEST=none

Change-Id: I36258bb21a0012b7625f579d00794ce37341f498
Reviewed-on: https://gerrit.chromium.org/gerrit/63224
Reviewed-by: Ryan Cui <rcui@chromium.org>
Commit-Queue: Jan Voung <jvoung@chromium.org>
Tested-by: Jan Voung <jvoung@chromium.org>
12 months agoAdd wolf configuration to cbuildbot_config. 04/62904/2
Mike Stipicevic [Mon, 22 Jul 2013 22:41:26 +0000 (15:41 -0700)]
Add wolf configuration to cbuildbot_config.

TEST=buildbot/cbuildbot_config_unittest.py
BUG=chromium:262272

Change-Id: Ie309bfab34e3c62363bedc6c7cc67fb7748f235c
Reviewed-on: https://gerrit.chromium.org/gerrit/62904
Tested-by: Mike Stipicevic <xusydoc@chromium.org>
Reviewed-by: Ryan Cui <rcui@chromium.org>
Commit-Queue: Mike Stipicevic <xusydoc@chromium.org>

12 months agoAdd the "message" subcommand to the gerrit command line tool 76/62776/3
Doug Anderson [Sat, 20 Jul 2013 15:30:24 +0000 (21:00 +0530)]
Add the "message" subcommand to the gerrit command line tool

This allows placing a message into the gerrit CL with the "message"
command.

BUG=chromium:254515
TEST=From command line, try:
  gerrit message 32556 "testing 1, 2, 3. \" \\ '"
See this added to gerrit:
  testing 1, 2, 3. " \ '

Change-Id: I667ba1791f16b4ec29612ba3efb2a7dbc3cb22f2
Signed-off-by: Doug Anderson <dianders@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/62776
Reviewed-by: David James <davidjames@chromium.org>
12 months agoDon't set -chrome_debug on perf builder. 23/62923/2
David James [Mon, 22 Jul 2013 23:37:27 +0000 (16:37 -0700)]
Don't set -chrome_debug on perf builder.

-chrome_debug disables the generation of debug symbols. We need debug
symbols in production so that they can be stripped and uploaded to the
crash server. We should do the same in perf builds to have more
representative builds, and also to allow developers to debug any
performance problems that are found in the tests.

BUG=none
TEST=All unit tests.

Change-Id: I5e15ef0ac32d89c6ffd39ab1d8c2a1ce17d8f168
Reviewed-on: https://gerrit.chromium.org/gerrit/62923
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoSet close_fds=False in setup_fds. 36/62736/3
David James [Fri, 19 Jul 2013 21:03:45 +0000 (14:03 -0700)]
Set close_fds=False in setup_fds.

I originally set close_fds=True here because I was looking to prevent
locks from bleeding into subprocesses. close_fds=True closes all file
descriptors except for the named file descriptors. This logic is no
longer necessary in new versions of Portage because Portage has added
its own logic for preventing bleeding locks.

close_fds=True is also a maintenance headache and causes issues with new
versions of Portage:

 ...
   File .../_emerge/PipeReader.py, line 42, in _start
     self._registered_events, output_handler))
   File .../portage/util/_eventloop/EventLoop.py, line 512, in io_add_watch
     self._poll_obj.register(f, condition)
   File  .../portage/util/_eventloop/EventLoop.py, line 633, in register
     self._epoll_obj.register(fd, *args)
 IOError: [Errno 22] Invalid argument

In older versions of Portage, the close_fds argument was not available,
so I've added a version check here.

BUG=chromium:218085
TEST=Many mario-paladin trybot runs.

Change-Id: I5ad83b4571eb6c00b15e2771a6287d4eb7ac79c5
Reviewed-on: https://gerrit.chromium.org/gerrit/62736
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoparallel_emerge: update emerge package api 27/43527/4
Mike Frysinger [Tue, 19 Feb 2013 20:53:46 +0000 (15:53 -0500)]
parallel_emerge: update emerge package api

Upstream has deprecated the .metadata member.

BUG=chromium:218085
TEST=paladin and full builds work on both the old and new versions of portage

Change-Id: I30e8ac38dbea787f126ecead2264262bb092c61b
Reviewed-on: https://gerrit.chromium.org/gerrit/43527
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoStrip underscores from version numbers 38/62238/2
Jim Hebert [Tue, 16 Jul 2013 22:43:17 +0000 (15:43 -0700)]
Strip underscores from version numbers

BUG=none
TEST=cros_extract_deps --format=cpe --board=x86-generic chromeos
TEST=diff output before/after this CL

Change-Id: If977ccd0b758077175bd4e7a17f1d1f449e57a31
Reviewed-on: https://gerrit.chromium.org/gerrit/62238
Tested-by: Jim Hebert <jimhebert@chromium.org>
Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org>
Commit-Queue: Jim Hebert <jimhebert@chromium.org>

12 months agoRemove TestFastPrinting as it's often flaky. 77/62577/2
David James [Fri, 19 Jul 2013 01:24:29 +0000 (18:24 -0700)]
Remove TestFastPrinting as it's often flaky.

Example flakiness:
  http://chromegw/i/chromeos/builders/mario%20paladin/builds/12216

BUG=chromium:261890
TEST=Run the unit tests

Change-Id: Ibffd2a53694b86709fad8ab842bb2e6728ad738d
Reviewed-on: https://gerrit.chromium.org/gerrit/62577
Reviewed-by: Marc MERLIN <merlin@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoDisable hw_tests for parrot32/parrot64 60/62560/3
Mohammed Habibulla [Thu, 18 Jul 2013 22:51:05 +0000 (15:51 -0700)]
Disable hw_tests for parrot32/parrot64

BUG=chromium:261838
TEST=none

Change-Id: I5219399401b8007b72aea38a47c87c3098f0eb12
Reviewed-on: https://gerrit.chromium.org/gerrit/62560
Commit-Queue: Mohammed Habibulla <moch@chromium.org>
Reviewed-by: Mohammed Habibulla <moch@chromium.org>
Tested-by: Mohammed Habibulla <moch@chromium.org>
12 months agocrosfw: Use /tmp as output directory by default 45/62545/3
Simon Glass [Thu, 18 Jul 2013 21:23:35 +0000 (15:23 -0600)]
crosfw: Use /tmp as output directory by default

Rather than pollute the U-Boot directory, put output files into /tmp.
Add an option to select a different directory if desired.

BUG=none
TEST=manual
crosfw -b sandbox -w
(puts files in /tmp/sandbox)

Set OUT_DIR = '/tmp/u-boot' in ~/.crosfw

crosfw -b sandbox -w
(puts files in /tmp/u-boot/sandbox)

Change-Id: I16c517c660df5234585091de1c8206695b57cf40
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/62545
Reviewed-by: Vadim Bendebury <vbendeb@chromium.org>
12 months agocrosfw: Correct case in comments about .crosfw variables 44/62544/2
Simon Glass [Thu, 18 Jul 2013 21:22:41 +0000 (15:22 -0600)]
crosfw: Correct case in comments about .crosfw variables

These settings should be in upper case.

BUG=none
TEST=none
Just a comment change.

Change-Id: Ic142faeb3d73c5d306c33b36a5bbf18c18437bcc
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/62544
Reviewed-by: Che-Liang Chiou <clchiou@chromium.org>
12 months agocrosfs: Fix missing result assignment 43/62543/2
Simon Glass [Thu, 18 Jul 2013 21:15:57 +0000 (15:15 -0600)]
crosfs: Fix missing result assignment

The -B flag when combined with -z fails due to this bug.

BUG=none
TEST=manual
crosfw -Bz

See that it no longer gives this error:

  File "/mnt/host/source/chromite/scripts/crosfw.py", line 480, in RunBuild
    if result.returncode:
UnboundLocalError: local variable 'result' referenced before assignment

Change-Id: Iaa71ce460cc8c99e8756a1b46ab38365b968b471
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/62543
Reviewed-by: Che-Liang Chiou <clchiou@chromium.org>
12 months agoEnsure completion stages run when 'stop build' is hit. 61/59361/4
David James [Tue, 18 Jun 2013 16:44:11 +0000 (09:44 -0700)]
Ensure completion stages run when 'stop build' is hit.

When a developer hits 'stop build' on a paladin builder, the master
paladin will wait forever. This is not the expected behavior. Similarly,
when Pre-CQ runs, if somebody stops the build, the CL will never be
approved by the Pre-CQ, and will be left waiting for a very long time
(the CQ will eventually pick it up when no other CLs are waiting.)

BUG=chromium:214624, chromium:246260
TEST=Run example remote trybot run and verify completion stage runs
     when you hit stop build
TEST=Run unit tests.
TEST=Run pre-cq with real change, hit stop build, and verify completion
     stage runs.
TEST=Run successful paladin run

CQ-DEPEND=CL:62429, CL:62427
Change-Id: Idea5587178ff0ac86a07bf38d238aeafa44b3071
Reviewed-on: https://gerrit.chromium.org/gerrit/59361
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoOnly run cleanup code when it's needed. 29/62429/3
David James [Thu, 18 Jul 2013 00:15:18 +0000 (17:15 -0700)]
Only run cleanup code when it's needed.

When EnforcedCleanupSection was written, the intention was that cleanup
sections would only be run in the child program if the parent program
was killed suddenly (e.g. kill -9) and didn't have a chance to run them.

Unfortunately, sharing state using a shared file descriptor doesn't
work. Therefore, I've switched the program to use pipes instead.

BUG=chromium:214624, chromium:246260
TEST=Run with CL:59361 and verify completion stage only run once.
TEST=Run example paladin run.

Change-Id: I9ac1c3f674ddf8a9b1ba7060418c3b864d875cb7
Reviewed-on: https://gerrit.chromium.org/gerrit/62429
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoDon't try to clean up the tee process. 27/62427/3
David James [Wed, 17 Jul 2013 23:45:05 +0000 (16:45 -0700)]
Don't try to clean up the tee process.

We don't want the critical section to try to clean up the tee process,
so we run Tee (forked off) outside of it. This prevents a deadlock
because the Tee process only exits when its pipe is closed, and the
critical section accidentally holds on to that file handle.

BUG=chromium:261402
TEST=Test on buildbot and verify hang is gone.
TEST=Paladin trybot run.

Change-Id: Ia457bbc9fe8c60de6b5c7474b5f490776a2cac41
Reviewed-on: https://gerrit.chromium.org/gerrit/62427
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoAdd 2 more Alex machines to the CQ pool. 96/62496/2
David James [Thu, 18 Jul 2013 17:28:06 +0000 (10:28 -0700)]
Add 2 more Alex machines to the CQ pool.

BUG=chromium:224817
TEST=All unit tests.

Change-Id: I3cf060acaa6d1b0bbb3a56df0486dc7a9416ac22
Reviewed-on: https://gerrit.chromium.org/gerrit/62496
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
12 months agoDon't cycle through an iterator twice. 63/62363/2
David James [Wed, 17 Jul 2013 17:46:41 +0000 (10:46 -0700)]
Don't cycle through an iterator twice.

Right now we attempt to cycle through an iterator twice, which obviously
doesn't work. Convert the iterator to a tuple so that we can cycle
through it twice without emptying our list.

BUG=chromium:258457
TEST=Unit tests. Master and slave paladin run.

Change-Id: Ia2a72e26baf89f1abecb04c8fd18055254d5d319
Reviewed-on: https://gerrit.chromium.org/gerrit/62363
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>

12 months ago[autotest] increase cq hwtest timeout from 50min to 60min 65/62365/3
Dan Shi [Wed, 17 Jul 2013 18:21:42 +0000 (11:21 -0700)]
[autotest] increase cq hwtest timeout from 50min to 60min

This is a temporary fix to get CQ hwtest running.

BUG=chromium:261258
TEST=none
DEPLOY=none

Change-Id: Iec8c6846ee87415caf7b12de1f89ef64a7bb1d66
Reviewed-on: https://gerrit.chromium.org/gerrit/62365
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
12 months agoTreat flake in UploadSymbols into a warning, not an error. 53/62353/2
David James [Wed, 17 Jul 2013 17:01:36 +0000 (10:01 -0700)]
Treat flake in UploadSymbols into a warning, not an error.

CL:58625 accidentally converted flake in UploadSymbols from a warning
to an error. This was a mistake, because the symbol upload server can
sometimes go down, and it's not reasonable to fail the whole build because
of this. Treat this as a warning rather than an error.

BUG=chromium:259652
TEST=Release trybot run.
TEST=Unit tests.

Change-Id: I89f3f66e32621765b33a272fa6e3b94263da64f1
Reviewed-on: https://gerrit.chromium.org/gerrit/62353
Reviewed-by: Ryan Cui <rcui@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoRun devinstall_test verbosely. 54/62354/2
Chris Sosa [Wed, 17 Jul 2013 17:03:39 +0000 (10:03 -0700)]
Run devinstall_test verbosely.

When this test fails, there's not enough info to go by when their
is flakiness. Turn on all logging until we are confident all flakiness
is gone.

BUG=chromium:260036
TEST=Ran it.

Change-Id: I70b390799079cc1e8dbb444e7e8068c041016b8d
Reviewed-on: https://gerrit.chromium.org/gerrit/62354
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Ryan Cui <rcui@chromium.org>
Commit-Queue: Chris Sosa <sosa@chromium.org>
Reviewed-by: Matt Tennant <mtennant@chromium.org>
12 months agoOnly submit changes if their dependencies were submitted. 72/60272/5
David James [Thu, 27 Jun 2013 21:34:00 +0000 (14:34 -0700)]
Only submit changes if their dependencies were submitted.

BUG=chromium:254209
TEST=Run modified unit test.
TEST=cbuildbot mario-paladin --buildbot --debug

Change-Id: I15c780873a4a8aaa3ce3d5a86f5457163a83a3b1
Reviewed-on: https://gerrit.chromium.org/gerrit/60272
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoAdd script for creating projects. 40/62140/4
David James [Tue, 16 Jul 2013 17:46:20 +0000 (10:46 -0700)]
Add script for creating projects.

The purpose of this script is to ensure that certain often overlooked
steps (specifying the parent for example) is done- in this case
automatically calculated if not specified- while specifying other
defaults.

This basically is a defaulted version of invoking gerrit
create-project, and should be the preferred norm going forward.

BUG=chromium:138826
TEST=Run the script in dry-run mode and verify the command line.

Change-Id: I2e7cc0aca5a68b0bfd7d7f55d7d0bce70961abb9
Reviewed-on: https://gerrit.chromium.org/gerrit/62140
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoAdd branching functionality to cbuildbot. 44/59144/17
Ryan Cui [Fri, 21 Jun 2013 18:35:30 +0000 (11:35 -0700)]
Add branching functionality to cbuildbot.

Adds a 'branch-util' config to cbuildbot for branch creation, rename,
and deletion.

Sample usages of the config:

cbuildbot branch-util --branch-name=release-R29-4319.B
--version=4319.0.0

cbuildbot branch-util --branch-name=release-R29-4319.B
--version=4319.0.0 --delete-branch

cbuildbot branch-util --branch-name=release-R29-4319.B
--version=4319.0.0 --rename-to='new-branch-name'

BUG=chromium:249566
TEST=Manually, unit tests, actually branching R29 and a stabilize
     branch.

Change-Id: Ie3e64048872c63ce4fe555ef114688bb3c42d395
Reviewed-on: https://gerrit.chromium.org/gerrit/59144
Reviewed-by: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
Commit-Queue: Ryan Cui <rcui@chromium.org>

12 months agocrosfw: Enable building beaglebone and VBOOT debugging 01/61501/2
Simon Glass [Wed, 10 Jul 2013 23:32:42 +0000 (16:32 -0700)]
crosfw: Enable building beaglebone and VBOOT debugging

The beaglebone has underscore in its name, so is not currently supported. The
fix is to only perform the Chrome OS underscore checking when we know it
is a Chrome OS board.

Also we should always enable VBOOT debugging, so correct that.

BUG=none
TEST=manual
$ crosfw -b am335x_evm -w
See that it now works correctly.
The VBOOT change was tested with a special U-Boot, CLs are still pending in
gerrit.

Change-Id: I19a73b4b0ed617c6a6384bd424858cac38fac374
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/61501
Reviewed-by: Randall Spangler <rspangler@chromium.org>
Reviewed-by: Che-Liang Chiou <clchiou@chromium.org>
12 months agoAdding file_bugs to stout canary run_suite invocation. 04/61804/2
beeps [Sat, 13 Jul 2013 00:33:35 +0000 (17:33 -0700)]
Adding file_bugs to stout canary run_suite invocation.

Only stout canaries are running with -f False.

TEST=Trybot similar change on lumpy-release.
BUG=None

Change-Id: I115873a88eace25ec9277f4a85aa42489aaded52
Reviewed-on: https://gerrit.chromium.org/gerrit/61804
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>

12 months agoAdd parrot32 and 64 paladin/canary. 28/61628/5
Mike Stipicevic [Fri, 12 Jul 2013 01:15:28 +0000 (18:15 -0700)]
Add parrot32 and 64 paladin/canary.

BUG=chromium:258156
TEST=Ran buildbot/cbuildbot_config_unittest.py.

Change-Id: I58c28664a24362e3e52ee861b371d5e97639901c
Reviewed-on: https://gerrit.chromium.org/gerrit/61628
Tested-by: Mike Stipicevic <xusydoc@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Mike Stipicevic <xusydoc@chromium.org>

12 months agoCompile everything from source (except Chrome) in the CQ. 69/61469/4
David James [Wed, 3 Jul 2013 22:40:37 +0000 (15:40 -0700)]
Compile everything from source (except Chrome) in the CQ.

This adds significant coverage to the commit queue without slowing down
the overall cycle time of the commit queue. It is possible to do this
because I've only updated targets that don't run hardware tests to compile
everything from source.

Testing of the patch suggests that the three bots I have updated will
rarely become the bottleneck of the CQ after being updated. The three
modified bots now take ~47 minutes. alex-paladin typically takes about 55
minutes to complete. The fastest time I've seen it finish in is 43 minutes,
which only happens when there are no packages to compile at all.

BUG=chromium:257167
TEST=Trybot runs of all commit queue targets.
TEST=Verify overall commit queue is not slowed down.

Change-Id: I90469c970512274712578138d6462c0de6248f4e
Reviewed-on: https://gerrit.chromium.org/gerrit/61469
Commit-Queue: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoHandle case where chromite dir is missing. 73/61373/3
David James [Tue, 9 Jul 2013 20:47:23 +0000 (13:47 -0700)]
Handle case where chromite dir is missing.

Right now cbuildbot uses cros_sdk before we've even updated the Chrome
OS source code. This can cause issues if the old version of cros_sdk
is missing or corrupted.

Fortunately, we already unmount the directories ourselves, so it is
safe for us to just rm -rf the dir.

BUG=chromium:258167
TEST=Run trybot run on machine with missing chromite dir.
TEST=Run a bunch of trybot runs including stop build runs.

Change-Id: I2b7d87588dafbac937e114225091c877702bd62c
Reviewed-on: https://gerrit.chromium.org/gerrit/61373
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agoUpdate format=cpe output to match consumer-side requirements. 22/61322/4
Jim Hebert [Tue, 9 Jul 2013 18:34:08 +0000 (11:34 -0700)]
Update format=cpe output to match consumer-side requirements.

This also adds sorting so that it's more deterministic and amenable
to diffing between runs.

BUG=none
TEST=cros_extract_deps --format=cpe --board=x86-generic chromeos

Change-Id: Ib5894c811c7747c53727fabd8086bfda13b720cd
Reviewed-on: https://gerrit.chromium.org/gerrit/61322
Tested-by: Jim Hebert <jimhebert@chromium.org>
Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org>
Commit-Queue: Jim Hebert <jimhebert@chromium.org>

12 months agoAdd call to test_au_zip.py to Unittest stage. 73/60173/6
Don Garrett [Thu, 27 Jun 2013 02:13:25 +0000 (19:13 -0700)]
Add call to test_au_zip.py to Unittest stage.

This new command to test the au_generator.zip is run outside the chroot. Added
to Unittest stage after normal tests complete.

BUG=chromium:214885
TEST=lint/unittests/trybot/trybot with a known invalid zip file.

Change-Id: I0f16b6382db55da5f336592dd813818d8dd360f0
Reviewed-on: https://gerrit.chromium.org/gerrit/60173
Commit-Queue: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
12 months agoSkip gerrit unittest on build servers. 09/61109/3
David James [Mon, 8 Jul 2013 17:59:04 +0000 (10:59 -0700)]
Skip gerrit unittest on build servers.

The gerrit unittest is bound to be flaky because it depends on gerrit
itself. Remove it from our list of automated tests to run on the bots.

BUG=chromium:258091
TEST=Confirm gerrit_unittest no longer runs from inside the chroot.

Change-Id: If320fa674a7bcbc850390747ebfbb31497c35ce7
Reviewed-on: https://gerrit.chromium.org/gerrit/61109
Reviewed-by: Chris Wolfe <cwolfe@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
12 months agochromite: Remove oem_wallpaper USE flag from butterfly. 32/60932/2
Daniel Erat [Wed, 3 Jul 2013 19:57:39 +0000 (12:57 -0700)]
chromite: Remove oem_wallpaper USE flag from butterfly.

BUG=chromium:248764
TEST=none

Change-Id: I6c6da6341a16549b25da8f0773eb1d56073cd635
Reviewed-on: https://gerrit.chromium.org/gerrit/60932
Reviewed-by: James Cook <jamescook@chromium.org>
Commit-Queue: Daniel Erat <derat@chromium.org>
Tested-by: Daniel Erat <derat@chromium.org>
12 months agocbuildbot_config: simplify adding new full configs 73/60573/3
Mike Frysinger [Sat, 29 Jun 2013 01:16:52 +0000 (21:16 -0400)]
cbuildbot_config: simplify adding new full configs

This makes it easier to add new full configs by updating a single list.
I also slipped in a few boards that were missing (link, lumpy, parrot,
zgb to name a few).

BUG=None
TEST=`./cbuildbot_config_unittest.py` passes
TEST=`./cbuildbot_config --dump` before & after just shows new bot configs added

Change-Id: Ia2017a6db13804e68404c64ebe360662d3e3f11e
Reviewed-on: https://gerrit.chromium.org/gerrit/60573
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
13 months agoEnable incrementing the Chrome branch in chromeos_version.sh 78/59578/2
Ryan Cui [Fri, 21 Jun 2013 00:44:32 +0000 (17:44 -0700)]
Enable incrementing the Chrome branch in chromeos_version.sh

To enable crbug.com/249566, add a 'chrome_branch' increment type to
manifest_versions.Version.

BUG=chromium:249566
TEST=unit tests.

Change-Id: I345f092e667a1f1bbd2f8253e633afe3d33e3c0f
Reviewed-on: https://gerrit.chromium.org/gerrit/59578
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>
13 months agoLocalize manifests for local trybot. 84/59584/3
Ryan Cui [Fri, 21 Jun 2013 18:35:12 +0000 (11:35 -0700)]
Localize manifests for local trybot.

Local trybots fail when running the ManifestVersionSyncStage because the
official manifests contain restricted projects.  To keep things working,
we filter out those projects in the sync stage.

BUG=None
TEST=locally, remote trybots.

Change-Id: I4a501e276beae5cf138337b3c0a1af627946a474
Reviewed-on: https://gerrit.chromium.org/gerrit/59584
Commit-Queue: Ryan Cui <rcui@chromium.org>
Reviewed-by: Ryan Cui <rcui@chromium.org>
Tested-by: Ryan Cui <rcui@chromium.org>