2 years agoFix long delays with binhost from behind proxy 10/49110/2 master cros-
Zac Medico [Tue, 23 Apr 2013 22:01:56 +0000 (15:01 -0700)]
Fix long delays with binhost from behind proxy

The version of portage in the chromeos build system is not
proxy-aware.  When running from behind a proxied firewall,
portage times-out when accessing URLs for binhost purposes,
creating very long delays when creating the chroot, setting
up boards, etc.  This backport of a patch from (upstream)
portage removes the attempt to use the obsolete binhost
protocol, removing the long delays.

Backported by Paul Drews <>

Original patch notes from upstream portage are as follows:

  | From: Zac Medico <>
  | Date: Wed, 17 Oct 2012 22:34:17 -0700
  | Subject: Remove old binhost protocol for bug #438640.
  | The old protocol has been deprecated since portage-2.1.6
  | (2009), so it should be completely irrelevant now.

TEST=behind proxied firewall: build chroot, setup_board

Change-Id: I729a295960b1698f8754ed60e162ab55c1b30bba
Reviewed-by: Mike Frysinger <>
Reviewed-by: Zac Medico <>
Tested-by: Mike Frysinger <>
2 years agorefactor contents file writing to its own function 50/47950/5 cros-
Aviv Keshet [Fri, 12 Apr 2013 00:51:34 +0000 (17:51 -0700)]
refactor contents file writing to its own function

This CL refactors out the functionality of writing a contents dicionary
to file. That functionality is required by a change in the chromeos
chromite repository.


Change-Id: I851724408b1d10827eee2ea8d67bdca9ad90c455
Commit-Queue: Zac Medico <>
Reviewed-by: Zac Medico <>
Tested-by: Zac Medico <>
Reviewed-by: Brian Harring <>
2 years agoAdd FEATURES=clean-logs support. cros-
Zac Medico [Wed, 31 Aug 2011 17:54:52 +0000 (10:54 -0700)]
Add FEATURES=clean-logs support.

Enable automatic execution of the command specified by the
PORT_LOGDIR_CLEAN variable. The default PORT_LOGDIR_CLEAN setting will
remove all files from PORT_LOGDIR that were last modified at least 7
days ago.
(cherry picked from commit 377720b203f2f22609da0429ec866ba671fc2da0)

TEST=`emerge portage` cleaned out my old log files from /var/log/portage

Change-Id: I99e4f9ca4f50081d3929f215099350f677e89e3c
Reviewed-by: Brian Harring <>
Commit-Ready: Mike Frysinger <>
Tested-by: Mike Frysinger <>
3 years agoxpak.unpackinfo: validate paths, bug #403149 cros-
Zac Medico [Sun, 12 Feb 2012 03:39:10 +0000 (19:39 -0800)]
xpak.unpackinfo: validate paths, bug #403149
(cherry picked from commit b3cfb2065ccbeb8f769d630ff997c0327fb2eb35)

3 years agoStrip .GCC.command.line from output files as well.
Michał Górny [Wed, 21 Dec 2011 16:20:38 +0000 (17:20 +0100)]
Strip .GCC.command.line from output files as well.
(cherry picked from commit 4e3a6eb60d48edddbe6a279f02b0cfc561394841)

3 years agoportageq mass_best_visible: optional "type" arg
Zdenek Behan [Fri, 13 Apr 2012 21:51:46 +0000 (14:51 -0700)]
portageq mass_best_visible: optional "type" arg
(cherry picked from 0bb454d0aecba0be1dd197038eefc4be6be2580e)

3 years agoApply INSTALL_MASK prior to reading the tree
Brian Harring [Fri, 27 Jan 2012 18:23:02 +0000 (10:23 -0800)]
Apply INSTALL_MASK prior to reading the tree

If this isn't done, collision-protect uses the old cached filelist,
leading to it complaining about things that INSTALL_MASK sorted already.

TEST=emerge sudo && emerge chromeos-base
(cherry picked from 02522e5755ab369c2d9a9f712bb0c1e7b882ad98)

3 years agoApply INSTALL_MASK prior to collision-protect.
Zac Medico [Sat, 14 Jan 2012 15:29:08 +0000 (07:29 -0800)]
Apply INSTALL_MASK prior to collision-protect.

It may be useful to avoid collisions in some scenarios.
(cherry picked from commit 93b654199a32fc3df1299b030317720b9294b0c3)

TEST=build_packages run
(cherry picked from 93b654199a32fc3df1299b030317720b9294b0c3)

3 years agoModify PKG_INSTALL_MASK to apply only to binpkg creation.
Brian Harring [Fri, 13 Jan 2012 04:13:42 +0000 (20:13 -0800)]
Modify PKG_INSTALL_MASK to apply only to binpkg creation.

Primarily useful for certain cases where cros needs to suppress debug
symbols being included for specific pkgs, and/or suppressing binpkgs
from carrying sensitive configuration data, but we still wish for it
to be installed.

TEST=set PKG_INSTALL_MASK, check ${D} vs what's in the binpkg
(cherry picked from 5d5364e982820fda56db5a40884e1c5f8e784b96)

3 years agosuppress splitdebug for .o files
Mike Frysinger [Thu, 20 Oct 2011 22:58:15 +0000 (15:58 -0700)]
suppress splitdebug for .o files

TEST=cbuildbot run of an arm builder, checking python.o after dump_syms
     runs against it

3 years agolayout.conf: add git friendly pregenerated cache format
Brian Harring [Fri, 14 Oct 2011 09:40:00 +0000 (02:40 -0700)]
layout.conf: add git friendly pregenerated cache format

Enabled via cache-format = md5-dict
This format is essentially just flat_hash, using md5 rather than mtime,
and dropping the path component from _eclasses_ entries.

From a speed standpoint, the md5 overhead is ~16% in comparison to mtime,
timed on a modern sandybridge; specifically, validating 29k nodes takes
~8.8s for flat_md5, while the pms norm is ~7.7s.

That said, the cache is /usable/ in places PMS is not; in those cases,
it can definitely be a win since even if the cache is partially old,
it's better than regenerating everything from scratch.
(cherry picked from commit 1e8870bd45a4e2a9c43e7f112701c6ae84b0fd56)

3 years agoegencache: add _mtime_ to metadata
Zac Medico [Sat, 15 Oct 2011 23:10:18 +0000 (16:10 -0700)]
egencache: add _mtime_ to metadata

This fixes a regression in FEATURES=parse-eapi-ebuild-head support
for egencache since commit 2ed1cb53cc4158af08c22d466b15b9a9a7767212.
If the metadata does not contain _mtime_ here, then _setitem from
cache.metadata.database will raise KeyError.
(cherry picked from commit 5e924b1c0467bdb6f2f9e517780844a07795b9da)

3 years agoperform_checksum: OSerror to PermissionDenied
Zac Medico [Mon, 17 Oct 2011 03:03:25 +0000 (20:03 -0700)]
perform_checksum: OSerror to PermissionDenied
(cherry picked from commit 5e2982d998ac5683612de4a776f92a0887a8839d)

3 years agohashed_path: convert OSError to PermissionDenied
Zac Medico [Mon, 17 Oct 2011 03:00:08 +0000 (20:00 -0700)]
hashed_path: convert OSError to PermissionDenied
(cherry picked from commit 60a2c22e1c40bb2e575320c5d30a5c23bb0a16bd)

3 years agohashed_path: convert stat OSError to FileNotFound
Zac Medico [Sun, 16 Oct 2011 21:13:45 +0000 (14:13 -0700)]
hashed_path: convert stat OSError to FileNotFound

This makes it consistent with perform_checksum call which also raises
FileNotFound instead of OSError.
(cherry picked from commit 5b8fdc7d3e4c64ce79d0218fd101f8292602d10b)

3 years agoFix a typo in a comment.
Arfrever Frehtes Taifersar Arahesis [Sun, 16 Oct 2011 12:49:58 +0000 (14:49 +0200)]
Fix a typo in a comment.
(cherry picked from commit c9e22480dbb1767cc0e59eee5c1c3b358da7a078)

3 years agoFix serialize_eclasses = False in cache template.
Zac Medico [Tue, 18 Oct 2011 05:26:27 +0000 (22:26 -0700)]
Fix serialize_eclasses = False in cache template.

This fixes volatile cache validation that did not get fixed in commit
a63c163a50bf8a4e5ca8673cd8bebae29c36643a. Now all volatile cache
problems that have been introduced since commit
2ed1cb53cc4158af08c22d466b15b9a9a7767212 should be fixed.
(cherry picked from commit 095a648f4726a467934b819e214a2468308ca351)

3 years agoportdbapi: fix volatile cache handling
Zac Medico [Mon, 17 Oct 2011 23:08:44 +0000 (16:08 -0700)]
portdbapi: fix volatile cache handling

The metadata_overlay usage, that was triggered automatically for users
without depcachedir write access, has been broken since commit
2ed1cb53cc4158af08c22d466b15b9a9a7767212. The metadata_overlay class
is not compatible with the new validation code, so remove it. If users
have metadata_overlay configured as their cache module, fall back to
(cherry picked from commit a63c163a50bf8a4e5ca8673cd8bebae29c36643a)

3 years agoeclass_cache: fix volatile cache DigestException
Zac Medico [Mon, 17 Oct 2011 21:42:09 +0000 (14:42 -0700)]
eclass_cache: fix volatile cache DigestException

The deepcopy in catch/ misbehaved when it tried to copy
attributes like __DEEPCOPY__ that didn't correspond to known functions.
(cherry picked from commit 58b1c71329f9d9ce0ee3a004d9ecaa8887d1dfd5)

3 years ago_pull_valid_cache: pass unicode to hashed_path
Zac Medico [Sun, 16 Oct 2011 05:58:56 +0000 (22:58 -0700)]
_pull_valid_cache: pass unicode to hashed_path

This removes a conversion to bytes that should have been removed in
commit 2ed1cb53cc4158af08c22d466b15b9a9a7767212. The hashed_path
location attribute should always be unicode, for consistency will path
handling throughout portage. The hashed_path class internally uses the
portage portage.os wrapper to ensure that paths are encoded as utf_8
bytes when they are passed to the real os module for stat calls.
(cherry picked from commit 28115d7d72cf0f0c42c636039ed2a08ee2412dd3)

3 years agoEbuildMetadataPhase: fix broken _metadata_callback
Zac Medico [Sat, 15 Oct 2011 22:46:37 +0000 (15:46 -0700)]
EbuildMetadataPhase: fix broken _metadata_callback

This fixes a regression in FEATURES=parse-eapi-ebuild-head support
for egencache since commit 2ed1cb53cc4158af08c22d466b15b9a9a7767212.
(cherry picked from commit 9f7aba55758ba93414ed07c58a0129178188c31b)

3 years agoportdbapi.aux_get: fix broken _metadata_callback
Zac Medico [Sat, 15 Oct 2011 20:28:44 +0000 (13:28 -0700)]
portdbapi.aux_get: fix broken _metadata_callback

This fixes a regression in FEATURES=parse-eapi-ebuild-head support
since commit 2ed1cb53cc4158af08c22d466b15b9a9a7767212.
(cherry picked from commit e3f0d11c8615010506052793689207b1c3b3bd86)

3 years agoemerge --metadata: fix empty INHERITED handling
Zac Medico [Sat, 15 Oct 2011 06:11:02 +0000 (23:11 -0700)]
emerge --metadata: fix empty INHERITED handling

Since changes to eclass validation code in commit
2ed1cb53cc4158af08c22d466b15b9a9a7767212, validation of cache for
packages with empty INHERITED metadata has been broken due to failure
to distinguish between None and empty dict return values from
(cherry picked from commit 0c1bbb316ce2e7e9608af78f5e2f5dea1c667af7)

3 years agoeclass_cache: fix cache_getter typo
Zac Medico [Sat, 15 Oct 2011 01:49:13 +0000 (18:49 -0700)]
eclass_cache: fix cache_getter typo

This triggered a regression since commit
2ed1cb53cc4158af08c22d466b15b9a9a7767212 that caused cache entries
containing eclass paths to appear invalid.
(cherry picked from commit 5e88b0d4ced1deb01ae18ea7868745d35b2fd7b2)

3 years agocache: rewrite to support arbitrary validation method
Brian Harring [Fri, 14 Oct 2011 06:27:22 +0000 (23:27 -0700)]
cache: rewrite to support arbitrary validation method

Specifically, the cache can use any portage supported checksum method,
or use the standard mtime approach.  In addition, support controlling
whether or not paths are stored, and generally try to restore some
of the centralization/encapsulation that was in place originally.
(cherry picked from commit bc1aed614fb588f0ade5bcb5d1265a8db0f8d247)

3 years agocache.util: drop completely dead module/code
Brian Harring [Fri, 14 Oct 2011 08:07:57 +0000 (01:07 -0700)]
cache.util: drop completely dead module/code

Looks of it, this code was copied/mangled into action_metadata; the
noise/callbacks in use don't do anything, so drop them, the dead code,
and clean up the exception handling to reflect the new reality
(cherry picked from commit 320890fadbedf678119ac06d458ed6086075431e)
(cherry picked from commit 90012f94f00ec2cf6b355bd9c30215b83c2b5b7f)

3 years ago_pull_valid_cache: fix readonly delete logic
Zac Medico [Fri, 14 Oct 2011 00:23:28 +0000 (17:23 -0700)]
_pull_valid_cache: fix readonly delete logic
(cherry picked from commit 382e74bb63d6f1eed2b3d88daee95fc7530df3da)

3 years agolayout.conf: make the pregenerated cache format controllable
Brian Harring [Thu, 13 Oct 2011 23:26:03 +0000 (16:26 -0700)]
layout.conf: make the pregenerated cache format controllable

Controllable via 'cache-format', currently it supports only one cache;
'pms', and defaults to it.  If an unsupported cache-format is specified,
the cache is disabled.  If pms is specified and metadata/cache directory
doesn't exist, the cache is disabled.

Finally, this rips out the best module support for locally overriding
the cache format used for pregenerated caches; this functionality made
zero sense (upstream determines the format, we use what is available).

(cherry picked from commit d4ea29bf6a3ce35d49e0f54f9173e3a6e42da2d6)

TEST=normal emerge invocations, egencache, etc.  Should be transparent.

3 years agodigestcheck: handle allow_missing for distfiles
Zac Medico [Thu, 13 Oct 2011 20:58:16 +0000 (13:58 -0700)]
digestcheck: handle allow_missing for distfiles
(cherry picked from commit a27f3f3058fdbdaca2bcb838aca02ffc4e32007a)

3 years agoprepstrip: make sure eu-strip embeds right filename
Mike Frysinger [Tue, 11 Oct 2011 17:28:45 +0000 (13:28 -0400)]
prepstrip: make sure eu-strip embeds right filename

Signed-off-by: Mike Frysinger <>
(cherry picked from commit 86e3ed5b8d1efdd439415cb3dec2905e5de058dc)

3 years agoprepstrip: add support for elfutils strip
Mike Frysinger [Tue, 11 Oct 2011 04:49:48 +0000 (00:49 -0400)]
prepstrip: add support for elfutils strip

If people use strip from the elfutils package, take advantage of some of
its neat features (like splitting + stripping in one step).

Signed-off-by: Mike Frysinger <>
(cherry picked from commit 51579fb34c19519a585ce2d052a270b6d84a2d97)

3 years agoprepstrip: extract buildid with readelf to avoid debugedit when possible
Mike Frysinger [Tue, 11 Oct 2011 04:40:09 +0000 (00:40 -0400)]
prepstrip: extract buildid with readelf to avoid debugedit when possible

The readelf utility is much more common than debugedit.

Signed-off-by: Mike Frysinger <>
(cherry picked from commit d4fdc3f5c04e556ce217e004ed461c9f05146c0f)

3 years agoprepstrip: avoid a `dirname` with a simple ${var%/*}
Mike Frysinger [Tue, 11 Oct 2011 02:28:55 +0000 (22:28 -0400)]
prepstrip: avoid a `dirname` with a simple ${var%/*}

Signed-off-by: Mike Frysinger <>
(cherry picked from commit 12dc5c56c821336f02338e3374b97e55273aee3c)

3 years agoprepstrip: tweak style of debugedit checks
Zac Medico [Mon, 10 Oct 2011 06:27:16 +0000 (23:27 -0700)]
prepstrip: tweak style of debugedit checks

Make the checks more like the FEATURES/RESTRICT checks from commit
(cherry picked from commit aeced1ac893951aec4992b2a1af7a4d057d6ce24)

3 years agoprepstrip: merge debugedit checks more
Zac Medico [Mon, 10 Oct 2011 06:04:33 +0000 (23:04 -0700)]
prepstrip: merge debugedit checks more

Use a ${debugedit_found} to remember if it's found or not, and use
${debugedit_warned} to avoid showing duplicate warnings.
(cherry picked from commit 13a759213ec1fefe5e637e5467fe9d5f35d8f570)

3 years agoprepstrip: optimize duplicate calls to `has`
Mike Frysinger [Mon, 10 Oct 2011 02:27:35 +0000 (22:27 -0400)]
prepstrip: optimize duplicate calls to `has`

Signed-off-by: Mike Frysinger <>
(cherry picked from commit 286675a500998e536dbbf2ee2dfe1fedf4290b38)

3 years agoprepstrip: merge debugedit checks
Mike Frysinger [Sun, 9 Oct 2011 22:53:53 +0000 (18:53 -0400)]
prepstrip: merge debugedit checks

Avoid checking for debugedit multiple times.

Signed-off-by: Mike Frysinger <>
(cherry picked from commit 2e6fa11b074d67ebeb799210256d4716d6655a97)

3 years agoprepstrip: optimize chmod slightly
Mike Frysinger [Sun, 9 Oct 2011 18:43:11 +0000 (14:43 -0400)]
prepstrip: optimize chmod slightly

Signed-off-by: Mike Frysinger <>
(cherry picked from commit 8ebca724748c0c2046e3120b0f5a813fc4298671)

3 years agoFix 'shopt: +s: invalid shell option name' error.
Arfrever Frehtes Taifersar Arahesis [Fri, 23 Sep 2011 03:23:01 +0000 (05:23 +0200)]
Fix 'shopt: +s: invalid shell option name' error.
(cherry picked from commit 13660734e81a8d35c7204fe05fb26068ad7bd19f)

3 years agomanifest: controllable per repo
Brian Harring [Fri, 23 Sep 2011 23:43:28 +0000 (16:43 -0700)]
manifest: controllable per repo

This adds three states to layout.conf key use-manifest; false, true, and strict.

false means "don't use manifests at all"
true means "use and generate manifests, but allow them to be missing"
strict means "manifests must be used everywhere in this repo"

TEST=repoman manifest usage.
(cherry picked from commit f3101b3adce6731790f80f83fafece54b7bd8a63)

3 years agoinstall_hooks: fix variable name for PORTAGE_CONFIGROOT
Fabian Groffen [Fri, 23 Sep 2011 18:35:29 +0000 (20:35 +0200)]
install_hooks: fix variable name for PORTAGE_CONFIGROOT

in addition, omit the leading slash, since PORTAGE_CONFIGROOT contains a
trailing slash
(cherry picked from commit ed9982d781a81651db0480af128a6509eafe358a)

3 years agoadd install hooks
Brian Harring [Wed, 21 Sep 2011 22:50:28 +0000 (15:50 -0700)]
add install hooks
(cherry picked from commit 9144182d9c8f0cf16973d8ec91eafc624310c6ca)

3 years agoman/portage.5: layout.conf examples manifests
Zac Medico [Tue, 13 Sep 2011 06:06:50 +0000 (23:06 -0700)]
man/portage.5: layout.conf examples manifests
(cherry picked from commit 68462daca775701f090cf9fc442310512c585e6b)

3 years agoRemove Manifest if it is not needed.
Zac Medico [Thu, 15 Sep 2011 02:36:54 +0000 (19:36 -0700)]
Remove Manifest if it is not needed.

With thin manifest, there's no need to have a Manifest file if there
are no DIST entries.
(cherry picked from commit 7a216218968dc1d00f2881121870611bc1b5dd33)

3 years agodoebuild: support allow-missing and thin manifest
Zac Medico [Tue, 13 Sep 2011 04:33:01 +0000 (21:33 -0700)]
doebuild: support allow-missing and thin manifest
(cherry picked from commit 48a2edbf0f0677cc97bae388cc391b563c8e4859)

3 years agoUpdate --rebuild-if-* flags to rebuild when build dependencies are changed.
David James [Mon, 22 Aug 2011 23:29:13 +0000 (16:29 -0700)]
Update --rebuild-if-* flags to rebuild when build dependencies are changed.

Right now, the --rebuild-if-* flags only rebuild packages that are used at
both run-time and build-time. This doesn't help for packages that are used
only at build-time (for example, static libaries).

Rebuilding packages whenever a build-time dependency is changed is easier to
understand and explain, and it handles all cases correctly.

TEST=Run emerge test suite.
(cherry picked from commit 2f253bcc364bd75c9b103133a17ea9887a3de15d)

3 years agoCommit fast-build patch to portage_tool repository.
David James [Tue, 13 Sep 2011 21:46:42 +0000 (14:46 -0700)]
Commit fast-build patch to portage_tool repository.

This patch allows us to selectively disable Portage locks in parallel_emerge.
It is not intended to be upstreamed but is left in for now until we find a way
to replace it (e.g. by integrating parallel emerge support directly into portage

Note- imported directly from patch, original metadata/sha1 lost

TEST=build_packages --nousepkg

Review URL:

3 years agoUpdate Portage to support fixing *.la and *.pc files, support suppressing env-update
David James [Thu, 15 Sep 2011 22:49:31 +0000 (15:49 -0700)]
Update Portage to support fixing *.la and *.pc files, support suppressing env-update

*.la and *.pc files need to be fixed up in order to ensure packages link
against the right library when we try to build / install to different ROOTs.

TEST=Compile with build_packages --nousepkg

3 years agoenv_update: add more vardbapi fallback code
Zac Medico [Sun, 18 Sep 2011 23:45:20 +0000 (16:45 -0700)]
env_update: add more vardbapi fallback code

For API consumers that call this function without the vardbapi
parameter, it whould continue to work correctly in all the cases
that worked before the vardbapi was added.
(cherry picked from commit 89664c0c4280372be5b593880c2ac5ccc695f688)

3 years agoenv_update: use global vardbapi as fallback
Zac Medico [Sun, 18 Sep 2011 22:47:45 +0000 (15:47 -0700)]
env_update: use global vardbapi as fallback
(cherry picked from commit 1a5a662780f7a4401a89f0939a9180872f36f05f)

3 years agomove locking into env_update itself
Brian Harring [Thu, 15 Sep 2011 22:06:44 +0000 (15:06 -0700)]
move locking into env_update itself
(cherry picked from commit 9623d8bfdeee4e2ab728529a86322a26f1b6258e)

3 years agodigestcheck: only show relevant msgs for thin
Zac Medico [Thu, 15 Sep 2011 05:30:46 +0000 (22:30 -0700)]
digestcheck: only show relevant msgs for thin
(cherry picked from commit 01d4dde5a17648b8004e3633bf11ed945a0fa796)

3 years agodigestcheck: remove empty/missing Manifest checks
Zac Medico [Thu, 15 Sep 2011 02:44:28 +0000 (19:44 -0700)]
digestcheck: remove empty/missing Manifest checks

These checks never really needed, and they are not valid for thin
manifests or allow-missing-manifests.
(cherry picked from commit 7b9709d96425606366b56b33168544a6897d69b0)

3 years agoDon't write empty (thin) Manifest files.
Zac Medico [Thu, 15 Sep 2011 02:21:45 +0000 (19:21 -0700)]
Don't write empty (thin) Manifest files.

With thin manifest, there's no need to have a Manifest file if there
are no DIST entries.
(cherry picked from commit d1f3fdfb943a9021d454c12b3418e44e5275ad69)

3 years agoadd layout.conf awareness of thin-manifests
Brian Harring [Thu, 1 Sep 2011 21:50:25 +0000 (14:50 -0700)]
add layout.conf awareness of thin-manifests

For any repo that wants thin (just src_uri digests), they just need to add

thin-manifests = true

to their layout.conf.  Again, this should only be used in repositories
were the backing vcs provides checksums for the ebuild data.
(cherry picked from commit af7933ee4df1d62a6567510dc7e84a0cf13a09ef)

3 years agoadd thin manifest support to the Manifest class
Brian Harring [Thu, 1 Sep 2011 21:36:45 +0000 (14:36 -0700)]
add thin manifest support to the Manifest class

'thin' is just distfiles.  This is primarily useful when the ebuild
lives in a vcs- git for example, which already has it's own checksums
to rely on.
(cherry picked from commit 69613e420d5be52e413c7c60e571710c2597f58d)

3 years agoBind all manifest access through repoconfigs
Brian Harring [Thu, 1 Sep 2011 00:29:58 +0000 (17:29 -0700)]
Bind all manifest access through repoconfigs

This enables controling the behaviour (creation and validation) per
repo, and while mildly ugly, refactors in the right direction.
(cherry picked from commit f908cddb505b81533861196c4713378e63dac1fa)

3 years agorefactoring; unhide part of the parsing functionality
Brian Harring [Wed, 31 Aug 2011 23:42:34 +0000 (16:42 -0700)]
refactoring; unhide part of the parsing functionality
(cherry picked from commit 53860ffa675b0cf1930589ff9fa15b5ffaa2cf75)

3 years agoRemove extended package set configuration docs. v2.1.10.11
Zac Medico [Fri, 3 Jun 2011 12:28:16 +0000 (05:28 -0700)]
Remove extended package set configuration docs.

Extended package sets configuration is disabled in the stable branch.

3 years agoRemove FEATURES=preserve-libs config/docs.
Zac Medico [Fri, 3 Jun 2011 12:15:50 +0000 (05:15 -0700)]
Remove FEATURES=preserve-libs config/docs.

Support for FEATURES=preserve-libs is disabled in the stable branch.

3 years agoportage.const: toggle constants for stable branch
Zac Medico [Fri, 3 Jun 2011 12:05:53 +0000 (05:05 -0700)]
portage.const: toggle constants for stable branch


3 years agorepoman: tweak --if-modified help output v2.2.0_alpha51
Zac Medico [Fri, 12 Aug 2011 11:43:40 +0000 (04:43 -0700)]
repoman: tweak --if-modified help output

3 years agorepoman: warn when --if-modified finds nothing
Zac Medico [Fri, 12 Aug 2011 11:21:53 +0000 (04:21 -0700)]
repoman: warn when --if-modified finds nothing

3 years agoFix log uid for logrotate-3.8 compat (bug 378451)
Zac Medico [Fri, 12 Aug 2011 09:47:04 +0000 (02:47 -0700)]
Fix log uid for logrotate-3.8 compat (bug 378451)

If PORT_LOGDIR is writable by the portage group but its uid is not
portage_uid, then set the uid to portage_uid if we have privileges to
do so, and also copy the uid to the logfile. This fixes logrotate
chown failures during the compression phase, when it attempts to copy
the uid from the logfile to a temp file. With the "su portage portage"
directive and logrotate-3.8.0, logrotate's chown call during the
compression phase will only succeed if the log file's uid is

3 years agoresume_depgraph: prune all indirect unsatisfied
Zac Medico [Thu, 11 Aug 2011 03:27:45 +0000 (20:27 -0700)]
resume_depgraph: prune all indirect unsatisfied

This should fix some cases of bug 378187.

3 years agorepoman: add --if-modified option to check less
Zac Medico [Thu, 11 Aug 2011 03:00:11 +0000 (20:00 -0700)]
repoman: add --if-modified option to check less

This is useful if you want to do a repo-level or category-level commit
but you only want to run checks for the packages that have uncommitted

3 years agorepoman: check more helper calls for $D, $ED...
Zac Medico [Tue, 9 Aug 2011 06:44:17 +0000 (23:44 -0700)]
repoman: check more helper calls for $D, $ED...

This adds docinto, docompress, fowners and fperms to the
variable.usedwithhelpers check. Notably absent is dosed since that
supports mixed input. This will fix bug #377303.

3 years agoPackage: remove unused profile.system mask type
Zac Medico [Tue, 9 Aug 2011 05:58:54 +0000 (22:58 -0700)]
Package: remove unused profile.system mask type

3 years agoRemove "profile" mask support as per PMS 5.2.7.
Zac Medico [Tue, 9 Aug 2011 05:35:38 +0000 (22:35 -0700)]
Remove "profile" mask support as per PMS 5.2.7.

Since this is considered a "legacy" feature by PMS 5.2.7, there
shouldn't be any need to support it anymore. Profiles can use
package.mask to get similar results, and package.mask additionally
allows for comments which are conveniently displayed by emerge. This
will fix bug #377907.

3 years ago_check_temp_dir: check for 'portage' symlink
Zac Medico [Tue, 9 Aug 2011 00:14:15 +0000 (17:14 -0700)]
_check_temp_dir: check for 'portage' symlink

This will fix bug #378403.

3 years agofetch: handle non-essential statvfs failure
Zac Medico [Sun, 7 Aug 2011 05:59:26 +0000 (22:59 -0700)]
fetch: handle non-essential statvfs failure

3 years agounmerge: remove more info dir files for #323213
Zac Medico [Sat, 6 Aug 2011 11:09:51 +0000 (04:09 -0700)]
unmerge: remove more info dir files for #323213

Treat any directory named "info" as a candidate here, since it might
have been in INFOPATH previously even though it may not be there now.

3 years agoportageq: fix short help display for leading \n
Zac Medico [Sat, 6 Aug 2011 05:16:31 +0000 (22:16 -0700)]
portageq: fix short help display for leading \n

3 years agounmerge: remove generated info "dir" files
Zac Medico [Thu, 4 Aug 2011 23:09:36 +0000 (16:09 -0700)]
unmerge: remove generated info "dir" files

These files are generated by emerge, so we need to remove them
when they are the only thing left in the directory. This will fix
bug #323213.

3 years agoaction_info: don't mix print with writemsg_stdout
Zac Medico [Thu, 4 Aug 2011 22:17:34 +0000 (15:17 -0700)]
action_info: don't mix print with writemsg_stdout

This should prevent strange output buffering/order issues reported when
piping output to head.

3 years agoportageq best_visible: support optional pkgtype
Zac Medico [Thu, 4 Aug 2011 21:26:12 +0000 (14:26 -0700)]
portageq best_visible: support optional pkgtype

3 years agoScheduler: fix parallel-fetch order with --jobs=1
Zac Medico [Wed, 3 Aug 2011 05:10:47 +0000 (22:10 -0700)]
Scheduler: fix parallel-fetch order with --jobs=1

This avoids a case where the first prefetcher is discarded, causing
the second prefetcher to occupy the fetch queue before the first
fetcher has an opportunity to execute.

3 years agounpack: remove backward compat for bug #376741 v2.2.0_alpha50
Zac Medico [Tue, 2 Aug 2011 18:25:58 +0000 (11:25 -0700)]
unpack: remove backward compat for bug #376741

This case is extremely rare, since the most common practice is to
unpack source files that are already in the cwd. The only Gentoo
packages that needed to be fixed were games-arcade/tuxpuck and

3 years agounmerge: display preserved syms in ascending ord
Zac Medico [Tue, 2 Aug 2011 08:56:11 +0000 (01:56 -0700)]
unmerge: display preserved syms in ascending ord

3 years agounpack: always unpack to cwd (bug #376741)
Zac Medico [Mon, 1 Aug 2011 23:48:07 +0000 (16:48 -0700)]
unpack: always unpack to cwd (bug #376741)

This brings portage into agreement with PMS. Also, for existing EAPIs,
if the source file is in a writable directory then this will create a
symlink for backward-compatible emulation of tools like gunzip and

3 years agodepgraph: fix duplicate zero pkg count display
Zac Medico [Mon, 1 Aug 2011 22:57:04 +0000 (15:57 -0700)]
depgraph: fix duplicate zero pkg count display

3 years agounmerge: unmerge symlinks broken by rmdir
Zac Medico [Mon, 1 Aug 2011 12:32:00 +0000 (05:32 -0700)]
unmerge: unmerge symlinks broken by rmdir

This prevents some more false alarms from commit

3 years agopost_emerge: just return instead of sys.exit()
Zac Medico [Mon, 1 Aug 2011 11:18:54 +0000 (04:18 -0700)]
post_emerge: just return instead of sys.exit()

3 years agounmerge: fix false alarm about protected symlinks v2.2.0_alpha49
Zac Medico [Fri, 29 Jul 2011 20:16:34 +0000 (13:16 -0700)]
unmerge: fix false alarm about protected symlinks

The error message about preserved symlinks, from commit
4bb08136f073024c5d31dceb1618b6f4e7246369, should only be triggered in
cases when the package being unmerged has the given path listed as a
symlink in its CONTENTS.

3 years agoFix a typo in a comment. v2.2.0_alpha48
Zac Medico [Fri, 29 Jul 2011 07:22:19 +0000 (00:22 -0700)]
Fix a typo in a comment.

3 years agoemerge: protect symlinks to directories sometimes
Zac Medico [Thu, 28 Jul 2011 11:29:25 +0000 (04:29 -0700)]
emerge: protect symlinks to directories sometimes

Before, it was possible to unmerge a symlink to a directory, such that
files installed via the path of the symlink could become inaccessible
via that path (and also making it impossible to unmerge them via that

Now, the symlink will only be unmerged if the directory that it points
to only contains regular files which are all being unmerged. In any
other case, the symlink will be preserved and an eerror log message
will record the event. This will give the user an opportunity to take
further action if they deem it necessary, and such symlink preservation
will not be silent as it was reported in bug #326685, comment #3.

3 years agodepgraph: handle invalid SRC_URI
Zac Medico [Wed, 27 Jul 2011 10:34:09 +0000 (03:34 -0700)]
depgraph: handle invalid SRC_URI

This is a minimal fix for bug #376577.

3 years agoconfig: fix AttributeError: 'NoneType' errors
Zac Medico [Wed, 27 Jul 2011 09:53:40 +0000 (02:53 -0700)]
config: fix AttributeError: 'NoneType' errors

This is a regression from commit
1fdeb78d08bac14f3f999f112499d179a87cf342 and it can only be triggered
when repo-level package.use is present.

3 years agomerge: abort if symlink replacing dir
Zac Medico [Wed, 27 Jul 2011 01:51:25 +0000 (18:51 -0700)]
merge: abort if symlink replacing dir

This is required for compliance with PMS section 13.4 as discussed in
bug #326685.

3 years agoNEWS: Add note about EAPI 3 in portage-
Zac Medico [Tue, 26 Jul 2011 00:44:31 +0000 (17:44 -0700)]
NEWS: Add note about EAPI 3 in portage-

3 years agodepgraph: make --exclude handle contradictions
Zac Medico [Mon, 25 Jul 2011 20:54:46 +0000 (13:54 -0700)]
depgraph: make --exclude handle contradictions

This makes contradictory things like `emerge -e @system --exclude gcc`
work as expected.

3 years agoFix a typo in the previous commit. v2.2.0_alpha47
Zac Medico [Sun, 24 Jul 2011 02:58:43 +0000 (19:58 -0700)]
Fix a typo in the previous commit.

3 years agoLinkageMapELF.getOwners(): add note about plibs
Zac Medico [Sun, 24 Jul 2011 02:57:52 +0000 (19:57 -0700)]
LinkageMapELF.getOwners(): add note about plibs

For preserved libraries, the owner(s) may have been been previously
uninstalled, but these uninstalled owners can be returned by this
method since they are registered in the PreservedLibsRegistry.

3 years agoLinkageMapELF: fix reversed x, cpv var refs
Zac Medico [Sun, 24 Jul 2011 02:14:52 +0000 (19:14 -0700)]
LinkageMapELF: fix reversed x, cpv var refs

This was an error in commit f393413c3f823ef4a60acfcc41c3920933510fc1.

3 years agoLibraryConsumerSet: handle KeyError from dbapi
Zac Medico [Sun, 24 Jul 2011 01:34:29 +0000 (18:34 -0700)]
LibraryConsumerSet: handle KeyError from dbapi

3 years agomerge: handle symlink replacing dir
Zac Medico [Sat, 23 Jul 2011 18:46:41 +0000 (11:46 -0700)]
merge: handle symlink replacing dir

Previously, these symlinks would trigger a file collision message and
would then be silently dropped in cases when the file collsion did not
cause emerge to bail out due to FEATURES=protect-owned or

Now, if emerge doesn't bail out due to a file collision, it will
trigger an eerror message and merge the symlink with a .backup.XXXX
extension appended to the file name, similar to handling of regular
files in commit 740f71301ed3daf44c0e77df5d5de39fe1438fb1. This will
fix bug 326685.

3 years agoLinkageMapELF: remove 5-tuple unpack code
Zac Medico [Sat, 23 Jul 2011 06:22:52 +0000 (23:22 -0700)]
LinkageMapELF: remove 5-tuple unpack code

3 years agoLinkageMapELF: add getOwners() method and use it
Zac Medico [Fri, 22 Jul 2011 22:59:13 +0000 (15:59 -0700)]
LinkageMapELF: add getOwners() method and use it

This preserves the owner information inside LinkageMap.rebuild() and
uses it to implement a getOwners() method, which makes it possible to
efficiently lookup owners of library providers and consumers.

3 years agoLinkageMapELF: use a class for _obj_properies
Zac Medico [Fri, 22 Jul 2011 20:38:04 +0000 (13:38 -0700)]
LinkageMapELF: use a class for _obj_properies

This simplifies the interface, avoiding the need for hardcoded indexes
and making it easy to add new attributes.

3 years agoLinkageMapELF: optimize memory usage
Zac Medico [Fri, 22 Jul 2011 07:58:56 +0000 (00:58 -0700)]
LinkageMapELF: optimize memory usage

This reduces memory consumption by approximately 30%, by replacing
mutable set instances with arrays, tuples, and frozensets where
appropriate. Also, identical frozenset instances are shared when