external/web-page-replay.git
5 months agoMake it possible to decompress content of type application/json git-svn master
rmcloughlin@google.com [Thu, 27 Feb 2014 17:59:25 +0000 (17:59 +0000)]
Make it possible to decompress content of type application/json

R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/183353003

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@546 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoFix broken command line parsing. Fix from avallee@chromium.org.
slamm@google.com [Fri, 21 Feb 2014 00:28:48 +0000 (00:28 +0000)]
Fix broken command line parsing. Fix from avallee@chromium.org.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@545 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoClean up the different uses of the word "path." From now on in a URL like http:...
rmcloughlin@google.com [Mon, 17 Feb 2014 15:28:12 +0000 (15:28 +0000)]
Clean up the different uses of the word "path."  From now on in a URL like host/abc?def=ghi, "path" always refers to "/abc" and "full path" always refers to "/abc?def=ghi"

R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/166253004

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@544 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoAdd an option to suppress socket errors. These don't cause any bad behavior but...
rmcloughlin@google.com [Mon, 17 Feb 2014 15:19:01 +0000 (15:19 +0000)]
Add an option to suppress socket errors.  These don't cause any bad behavior but they do clog up the logs.

R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/168713002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@543 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoFix mistake in find_closest_request docstring.
rmcloughlin@google.com [Thu, 13 Feb 2014 18:48:32 +0000 (18:48 +0000)]
Fix mistake in find_closest_request docstring.

R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/163473004

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@542 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoMake it possible to instantiate an ArchivedHttpRequest with path=None. Needed to...
rmcloughlin@google.com [Wed, 12 Feb 2014 16:30:06 +0000 (16:30 +0000)]
Make it possible to instantiate an ArchivedHttpRequest with path=None.  Needed to fix broken unit test in httparchive_test.py.

R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/156603008

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@541 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoMake traffic shaping work when WPR chooses the server ports.
tonyg@google.com [Tue, 4 Feb 2014 18:08:07 +0000 (18:08 +0000)]
Make traffic shaping work when WPR chooses the server ports.

WPR may be invoked with ports of 0 to let WPR choose an ephemeral port.
However, we weren't handling this properly and detecting the chosen port
for traffic shaping. This CL fixes that by getting the actual port used.

R=slamm@chromium.org

Review URL: https://codereview.chromium.org/153503002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@540 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoFix typo
tonyg@google.com [Tue, 4 Feb 2014 02:39:25 +0000 (02:39 +0000)]
Fix typo

TBR=slamm@chromium.org

Review URL: https://codereview.chromium.org/146913006

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@539 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoAllow executables to be found even when run as root.
tonyg@google.com [Mon, 3 Feb 2014 23:44:27 +0000 (23:44 +0000)]
Allow executables to be found even when run as root.

This fixes:
AssertionError: Failed to find ipfw in path

R=simonhatch@chromium.org

Review URL: https://codereview.chromium.org/147273004

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@538 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoFix rules about when root is required and address slamm's comments from previous CL.
tonyg@google.com [Sat, 1 Feb 2014 01:01:06 +0000 (01:01 +0000)]
Fix rules about when root is required and address slamm's comments from previous CL.

TBR=slamm@chromium.org

Review URL: https://codereview.chromium.org/137623012

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@537 164481a5-15cb-f69f-4b93-856c5b7754c5

5 months agoMake WPR more permission friendly.
tonyg@google.com [Thu, 30 Jan 2014 17:24:38 +0000 (17:24 +0000)]
Make WPR more permission friendly.

This makes it so that we don't have to run the whole script under sudo. Each
sudo-requiring command can be run with sudo and the user can be prompted to
remember the credentials.

This also makes WPR more chrome buildbot friendly.

R=jeremy@chromium.org

Review URL: https://codereview.chromium.org/146273003

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@536 164481a5-15cb-f69f-4b93-856c5b7754c5

6 months agoFix invalid reset of DNS source on network interface on Windows.
tonyg@google.com [Thu, 23 Jan 2014 17:51:35 +0000 (17:51 +0000)]
Fix invalid reset of DNS source on network interface on Windows.

Running replay.py on windows override the dhcp option.

On windows, the DNS property can be set to 'dhcp'.
An IP for a DNS server will automatically be found.

Here is the output before running replay.py:

C:\src\webpagereplay>netsh interface ip show dns

Configuration for interface "Wireless Network Connection"
DNS servers configured through DHCP: 192.168.2.1
Register with which suffix: Primary only

And the same command after...

Configuration for interface "Wireless Network Connection"
Statically Configured DNS Servers: 192.168.2.1
Register with which suffix: Primary only

It is not enough to restore the IP address. The DHCP/Static flag must also be saved/restored.

This patch uses 'netsh set dns' to save/restore the DHCP configuration.

Here is the verbose output of an execution:

C:\src\webpagereplay>python replay.py -C dummy.wpr -l debug
WARNING: sdpy_common: import c_zlib failed. Using uncompressed headers.
2014-01-23 00:27:16,549 DEBUG netsh interface ip show dns
2014-01-23 00:27:16,641 INFO Saved original primary DNS nameserver: 192.168.2.1
2014-01-23 00:27:16,642 INFO Loaded 1 responses from dummy.wpr
2014-01-23 00:27:16,644 DEBUG netsh interface ip show dns
2014-01-23 00:27:16,713 DEBUG netsh interface ip set dns Wireless Network Connection static 127.0.0.1
2014-01-23 00:27:16,792 DEBUG netsh interface ip show dns
2014-01-23 00:27:16,871 INFO Changed temporary primary nameserver to 127.0.0.1
2014-01-23 00:27:16,872 WARNING DNS server started on 127.0.0.1:53
2014-01-23 00:27:19,036 WARNING HTTP server started on 127.0.0.1:80
2014-01-23 00:27:19,109 WARNING HTTPS server started on 127.0.0.1:443
2014-01-23 00:27:21,943 DEBUG dnsproxy: teredo.ipv6.microsoft.com. -> 127.0.0.1 (replay web proxy)
2014-01-23 00:27:24,269 DEBUG dnsproxy: teredo.ipv6.microsoft.com. -> 127.0.0.1 (replay web proxy)
2014-01-23 00:27:25,796 DEBUG _real_dns_lookup(ipv6.msftncsi.com.) -> None (NoAnswer)
2014-01-23 00:27:25,796 DEBUG dnsproxy: ipv6.msftncsi.com. -> NXDOMAIN
2014-01-23 00:27:37,634 DEBUG _real_dns_lookup(wpad.gateway.2wire.net.) -> No nameserver.
2014-01-23 00:27:37,634 DEBUG dnsproxy: wpad.gateway.2wire.net. -> NXDOMAIN
2014-01-23 00:27:40,224 DEBUG dnsproxy: update.microsoft.com. -> 127.0.0.1 (replay web proxy)
2014-01-23 00:27:55,188 INFO Stopped HTTPS server. Total time processing requests: 0ms
2014-01-23 00:27:55,545 INFO Stopped HTTP server. Total time processing requests: 0ms
2014-01-23 00:27:55,690 INFO Stopped DNS server
2014-01-23 00:27:55,690 INFO Shutting down.
2014-01-23 00:27:55,690 DEBUG netsh interface ip set dns Wireless Network Connection static 192.168.2.1
2014-01-23 00:27:56,848 DEBUG netsh interface ip set dns Wireless Network Connection dhcp

Contributed by: Etienne Bergeron (etienneb@chromium.org)

R=nduca@chromium.org,tonyg@chromium.org

Review URL: https://codereview.chromium.org/142793003

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@535 164481a5-15cb-f69f-4b93-856c5b7754c5

6 months agoSimplify dns resolving error reporting.
tonyg@google.com [Thu, 23 Jan 2014 06:36:01 +0000 (06:36 +0000)]
Simplify dns resolving error reporting.

On startup, Chrome send a few invalid DNS requests.
When recording a page, this produce a lots of stack trace to the console.

With this patch, invalid DNS requests are display like that:

2014-01-22 21:21:40,490 DEBUG _real_dns_lookup(dvoraxidxk.gateway.2wire.net.) -> No nameserver.
2014-01-22 21:21:40,490 DEBUG dnsproxy: dvoraxidxk.gateway.2wire.net. -> NXDOMAIN
2014-01-22 21:21:40,509 DEBUG _real_dns_lookup(upursecmjv.gateway.2wire.net.) -> No nameserver.
2014-01-22 21:21:40,509 DEBUG dnsproxy: upursecmjv.gateway.2wire.net. -> NXDOMAIN
2014-01-22 21:21:40,526 DEBUG _real_dns_lookup(effuyfmmym.gateway.2wire.net.) -> No nameserver.
2014-01-22 21:21:40,526 DEBUG dnsproxy: effuyfmmym.gateway.2wire.net. -> NXDOMAIN

instead of

Exception happened during processing of request from ('127.0.0.1', 55081)
Traceback (most recent call last):
File "c:\src\depot_tools\python276_bin\lib\SocketServer.py", line 593, in process_request_thread
self.finish_request(request, client_address)
File "c:\src\depot_tools\python276_bin\lib\SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "c:\src\depot_tools\python276_bin\lib\SocketServer.py", line 649, in __init__
self.handle()
File "C:\src\webpagereplay\dnsproxy.py", line 202, in handle
ip = self.server.dns_lookup(self.domain)
File "C:\src\webpagereplay\dnsproxy.py", line 107, in __call__
ip = f(hostname, default_ip=ip)
File "C:\src\webpagereplay\dnsproxy.py", line 141, in __call__
real_ip = self.real_dns_lookup(host)
File "C:\src\webpagereplay\dnsproxy.py", line 84, in __call__
if answers:

Contributed by: Etienne Bergeron (etienneb@chromium.org)

Review URL: https://codereview.chromium.org/143133003

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@534 164481a5-15cb-f69f-4b93-856c5b7754c5

6 months agoAdd a web-page-replay-log command. Use it to post a message that WPR will log.
rmcloughlin@google.com [Fri, 17 Jan 2014 18:43:16 +0000 (18:43 +0000)]
Add a web-page-replay-log command.  Use it to post a message that WPR will log.

R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/140503003

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@533 164481a5-15cb-f69f-4b93-856c5b7754c5

6 months agoFixes invalid arguments to logging function.
tonyg@google.com [Thu, 16 Jan 2014 19:02:10 +0000 (19:02 +0000)]
Fixes invalid arguments to logging function.

Without this fix, chrome telemetry fails on windows.

Contributed by: Etienne Bergeron (etienneb@chromium.org)

R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/141023004

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@532 164481a5-15cb-f69f-4b93-856c5b7754c5

6 months ago_manage_request_stats decorator was buggy ('request' not defined). Put the code...
rmcloughlin@google.com [Thu, 16 Jan 2014 17:14:07 +0000 (17:14 +0000)]
_manage_request_stats decorator was buggy ('request' not defined).  Put the code from it into the one method it was decorating.

R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/139293008

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@531 164481a5-15cb-f69f-4b93-856c5b7754c5

6 months agoAdd missing import to httpproxy.py
rmcloughlin@google.com [Wed, 15 Jan 2014 22:14:37 +0000 (22:14 +0000)]
Add missing import to httpproxy.py

Review URL: https://codereview.chromium.org/137443016

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@530 164481a5-15cb-f69f-4b93-856c5b7754c5

7 months agoTeach Web Page Replay to handle all HTTP methods.
tonyg@google.com [Sun, 15 Dec 2013 21:39:56 +0000 (21:39 +0000)]
Teach Web Page Replay to handle all HTTP methods.

Previously it only handled GET, POST, and HEAD.

BUG=https://code.google.com/p/web-page-replay/issues/detail?id=33
R=slamm@chromium.org

Review URL: https://codereview.chromium.org/115903002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@529 164481a5-15cb-f69f-4b93-856c5b7754c5

8 months agoSome reliability improvements for the DNS server borrowed from the HTTP servers.
tonyg@google.com [Fri, 15 Nov 2013 16:12:45 +0000 (16:12 +0000)]
Some reliability improvements for the DNS server borrowed from the HTTP servers.

R=slamm@chromium.org

Review URL: https://codereview.chromium.org/73683002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@528 164481a5-15cb-f69f-4b93-856c5b7754c5

8 months agoFactor network configurations out to a new file so Telemetry can share them.
tonyg@google.com [Thu, 14 Nov 2013 17:54:18 +0000 (17:54 +0000)]
Factor network configurations out to a new file so Telemetry can share them.

Also add dialup and 3g.

R=slamm@chromium.org

Review URL: https://codereview.chromium.org/63173019

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@527 164481a5-15cb-f69f-4b93-856c5b7754c5

8 months agoAdd override for crypto.getRandomValues
slamm@google.com [Tue, 12 Nov 2013 21:19:22 +0000 (21:19 +0000)]
Add override for crypto.getRandomValues

BUG=
R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/66733005

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@526 164481a5-15cb-f69f-4b93-856c5b7754c5

8 months agoAllow WPR to chose the ports itself.
tonyg@google.com [Mon, 11 Nov 2013 18:08:01 +0000 (18:08 +0000)]
Allow WPR to chose the ports itself.

1. Binding to port 0 causes the OS to chose the port. And it doesn't require
   admin.
2. Clearly reporting the chosen port in the stdout allows the caller to scrape
   it.

This is in support of https://codereview.chromium.org/67983002/

R=slamm@chromium.org

Review URL: https://codereview.chromium.org/68113002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@525 164481a5-15cb-f69f-4b93-856c5b7754c5

8 months agoweb_page_replay: Stop fatally abort when PIL isn't available
tonyg@google.com [Tue, 5 Nov 2013 03:19:55 +0000 (03:19 +0000)]
web_page_replay: Stop fatally abort when PIL isn't available

Contributed by: Oystein Eftevaag (oysteine@chromium.org)

R=tonyg@chromium.org
NOTRY=true
BUG=

Review URL: https://codereview.chromium.org/56973013

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@524 164481a5-15cb-f69f-4b93-856c5b7754c5

9 months ago[web page replay] Make script injector work with long comments and without <head...
chrisgao@chromium.org [Wed, 16 Oct 2013 18:20:58 +0000 (18:20 +0000)]
[web page replay] Make script injector work with long comments  and without <head> and <html> tag.

R=tonyg@chromium.org

Review URL: https://codereview.chromium.org/23969009

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@522 164481a5-15cb-f69f-4b93-856c5b7754c5

10 months ago[Web Page Replay] Factor out script_injector.
tonyg@google.com [Mon, 9 Sep 2013 16:26:18 +0000 (16:26 +0000)]
[Web Page Replay] Factor out script_injector.

Contributed by: Shuotao Gao (chrisgao@chromium.org)

Review URL: https://codereview.chromium.org/23684040

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@521 164481a5-15cb-f69f-4b93-856c5b7754c5

11 months ago[webpagereplay] Add command exit.
tonyg@google.com [Wed, 28 Aug 2013 01:39:23 +0000 (01:39 +0000)]
[webpagereplay] Add command exit.

This is to make web page replay exits normally with graceful cleanup like restoring dns.

Contributed by: Shuotao Gao (chrisgao@chromium.org)

Review URL: https://codereview.chromium.org/23662002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@520 164481a5-15cb-f69f-4b93-856c5b7754c5

11 months agoAdd --host argument to replay.py to allow non-loopback interfaces
tonyg@google.com [Thu, 15 Aug 2013 20:49:46 +0000 (20:49 +0000)]
Add --host argument to replay.py to allow non-loopback interfaces

This is necessary to support RNDIS-based forwarding from Android.

Contributed By: Szymon Jakubczak (szym@chromium.org)

BUG=https://code.google.com/p/chromium/issues/detail?id=272381

Review URL: https://codereview.chromium.org/23076005

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@519 164481a5-15cb-f69f-4b93-856c5b7754c5

12 months agoMake WPR passthrough requests to other ports on the same machine.
tonyg@google.com [Mon, 8 Jul 2013 18:04:27 +0000 (18:04 +0000)]
Make WPR passthrough requests to other ports on the same machine.

This allows another web server on the same machine to serve requests. It is
useful for Telemetry which runs a static HTTP server for serving files on the
local filesystem. But chrome is running with --testing-fixed-http-port, so all
localhost requests get directed to WPR's port.

Anyway, we can now have Telemetry PageSets with both file:// and http:// URLs
where the http:// will be handled by WPR and file:// by the static server.

BUG=http://code.google.com/p/chromium/issues/detail?id=257209
R=slamm@google.com

Review URL: https://codereview.chromium.org/18713002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@518 164481a5-15cb-f69f-4b93-856c5b7754c5

12 months agoReduce total response time by 37%.
tonyg@google.com [Mon, 8 Jul 2013 17:52:34 +0000 (17:52 +0000)]
Reduce total response time by 37%.

Previously, get_requests() would linearly search through every request in the
archive looking for potential matches. This patch keeps an additional map of
hostname to {request: response} dicts. That way we only have to call matches()
for requests of the same hostname. With this change in place, get_requests() is
always 0 or 1ms in my local testing.

This exposed a bug where matches() was not properly comparing is_ssl. I fixed
that.

This change also makes future optimization easier by:
1. Adding an @LogRunTime annotate which logs the runtime of the annotated
   function.
2. Print the total request processing time at server exit.

Before:
013-07-04 09:28:07,055 INFO Stopped HTTPS server. Total time processing requests: 230ms
2013-07-04 09:28:07,207 INFO Stopped HTTP server. Total time processing requests: 1611ms

After:
2013-07-04 09:29:35,513 INFO Stopped HTTPS server. Total time processing requests: 234ms
2013-07-04 09:29:35,732 INFO Stopped HTTP server. Total time processing requests: 1011ms

R=slamm@google.com

Review URL: https://codereview.chromium.org/18714002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@517 164481a5-15cb-f69f-4b93-856c5b7754c5

12 months agoAvoid calling ratio() for many of the response candidates.
tonyg@google.com [Mon, 1 Jul 2013 20:31:35 +0000 (20:31 +0000)]
Avoid calling ratio() for many of the response candidates.

This speeds up premierleague.com in the typical_25 page set by almost 20%. It is
still guaranteed to find the candidate with the maximum ratio(), unlike my
previous attempt of only calling quick_ratio().

Before:
RESULT times_by_url: http___premierleague.com= [2914,2217,1957,1994,1937] ms
Avg times_by_url: 2203.800000ms
Sd  times_by_url: 412.512666ms

After:
RESULT times_by_url: http___premierleague.com= [2940,1678,1513,1425,1445] ms
Avg times_by_url: 1800.200000ms
Sd  times_by_url: 644.884253ms

R=slamm@google.com

Review URL: https://codereview.chromium.org/18176003

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@516 164481a5-15cb-f69f-4b93-856c5b7754c5

13 months agoRevert change from ratio() to quick_ratio().
tonyg@google.com [Fri, 21 Jun 2013 18:06:09 +0000 (18:06 +0000)]
Revert change from ratio() to quick_ratio().

This broke the ability to login to google accounts on android. See
http://crbug.com/252786

TBR=slamm@google.com

Review URL: https://codereview.chromium.org/17256003

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@515 164481a5-15cb-f69f-4b93-856c5b7754c5

13 months agoFix logic error in rerun_as_administrator.
tonyg@google.com [Fri, 7 Jun 2013 21:44:07 +0000 (21:44 +0000)]
Fix logic error in rerun_as_administrator.

BUG=29
R=slamm@google.com

Review URL: https://codereview.chromium.org/15841012

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@514 164481a5-15cb-f69f-4b93-856c5b7754c5

13 months agoMake WPR over 4 times faster and more stable.
tonyg@google.com [Mon, 3 Jun 2013 17:16:18 +0000 (17:16 +0000)]
Make WPR over 4 times faster and more stable.

1. Cache the results of formatted_request_line and path_without_query.
2. In find_closest_request, call quick_ratio() instead of ratio() and
   avoid calling it entirely when there is only one possible request.
3. Make matches many ties faster by bailing out earlier and doing
   less work in urlparse.

Here are some page load times for a replaying recording of linkedin.

Without patch (cold page loads):
*RESULT times: page_load_time= [1351,4144,2438,1844,1903,1845,1793,1793,1780,1726] ms
Avg times: 2061.700000ms
Sd  times: 777.364215ms

With patch (cold page loads):
*RESULT times: page_load_time= [920,546,560,528,449,363,513,337,452,330] ms
Avg times: 499.800000ms
Sd  times: 170.688670ms

Without patch (warm page loads):
*RESULT times: page_load_time= [2163,937,1617,1738,1718,1624,1584,1638,1520,1566] ms
Avg times: 1610.500000ms
Sd  times: 297.867327ms

With patch (warm page loads):
*RESULT times: page_load_time= [906,261,212,227,211,198,178,183,191,178] ms
Avg times: 274.500000ms
Sd  times: 223.355845ms

R=slamm@google.com

Review URL: https://codereview.chromium.org/16315002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@513 164481a5-15cb-f69f-4b93-856c5b7754c5

14 months agoAdd x-chrome-variations to the list of "undesirable headers"
tonyg@google.com [Wed, 8 May 2013 15:41:15 +0000 (15:41 +0000)]
Add x-chrome-variations to the list of "undesirable headers"

Contributed by: Ryan McLoughlin (rmcloughlin@google.com)

Review URL: https://codereview.chromium.org/14667010

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@512 164481a5-15cb-f69f-4b93-856c5b7754c5

15 months agoFix race in persisting httparchive after recording.
tonyg@google.com [Tue, 16 Apr 2013 16:41:42 +0000 (16:41 +0000)]
Fix race in persisting httparchive after recording.

When the http server shuts down, its daemon threads may still be living and
there is a chance that they could still write to the httparchive dict. When this
happens during pickling, it raises:
RuntimeError: dictionary changed size during iteration

This change prevents other threads from interrupting and updating the dict
during iteration.

Review URL: https://codereview.chromium.org/13991011

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@511 164481a5-15cb-f69f-4b93-856c5b7754c5

16 months agoBug fix: set trueport to None if host header doesn't contain a port number.
tonyg@google.com [Wed, 13 Mar 2013 17:22:08 +0000 (17:22 +0000)]
Bug fix: set trueport to None if host header doesn't contain a port number.

Contributed by: Ryan McLoughlin (rmcloughlin@google.com)

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@510 164481a5-15cb-f69f-4b93-856c5b7754c5

17 months agoAccept host headers in the form hostname:port
tonyg@google.com [Fri, 22 Feb 2013 22:44:34 +0000 (22:44 +0000)]
Accept host headers in the form hostname:port

Contributed by: Ryan McLoughlin (rmcloughlin@google.com)

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@509 164481a5-15cb-f69f-4b93-856c5b7754c5

17 months agoStart http & https proxies on 0.0.0.0 when in server mode.
tonyg@google.com [Thu, 21 Feb 2013 19:14:19 +0000 (19:14 +0000)]
Start http & https proxies on 0.0.0.0 when in server mode.

Contributed by: Ryan McLoughlin (mcloughlin@google.com)

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@508 164481a5-15cb-f69f-4b93-856c5b7754c5

19 months agoAdd a deterministic getTimezoneOffset() method.
tonyg@google.com [Wed, 19 Dec 2012 21:20:42 +0000 (21:20 +0000)]
Add a deterministic getTimezoneOffset() method.

The real getTimezoneOffset() depends on the timezone of the machine. GMail
doesn't replay properly unless the timezone is the same as it was during the
recording. A deterministic implementation is necessary to replay on bots with
a timezone different than where the original recording was made.

This implementation fakes PST/PDT for the year 2010. That makes Closure's
TimeZoneDetector happy. For other years, the method is still deterministic, but
we don't bother to simulate DST there (as that would require hardcoding a
bunch of dates).
Review URL: https://codereview.chromium.org/11612029

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@506 164481a5-15cb-f69f-4b93-856c5b7754c5

20 months agoFixes a problem that keeps the /web-page-replay-command-status command from working...
tonyg@google.com [Tue, 20 Nov 2012 16:44:52 +0000 (16:44 +0000)]
Fixes a problem that keeps the /web-page-replay-command-status command from working properly if the archive file is empty.

Patch contributed by: rmcloughlin@google.com

Issue=http://code.google.com/p/web-page-replay/issues/detail?id=28

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@505 164481a5-15cb-f69f-4b93-856c5b7754c5

20 months agoAugment command 'status' custom handler with CLI options and archive stats.
tonyg@google.com [Fri, 2 Nov 2012 23:17:04 +0000 (23:17 +0000)]
Augment command 'status' custom handler with CLI options and archive stats.

Patch contributed by: zeinwali@gmail.com

BUG=
Review URL: http://codereview.chromium.org/11292002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@502 164481a5-15cb-f69f-4b93-856c5b7754c5

21 months agoFix replaying on Windows.
tonyg@google.com [Tue, 23 Oct 2012 21:50:18 +0000 (21:50 +0000)]
Fix replaying on Windows.

1. Instead of running openssl at runtime to generate the cert, just check in a
   generated certfile.
2. Bind to localhost only. This fixes a security dialog prompt.
Review URL: https://codereview.chromium.org/11232070

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@501 164481a5-15cb-f69f-4b93-856c5b7754c5

21 months agoThis partially answers a request in http://codereview.chromium.org/9618036/.
tonyg@google.com [Mon, 22 Oct 2012 21:13:29 +0000 (21:13 +0000)]
This partially answers a request in codereview.chromium.org/9618036/.
The Json response can be further used as a response to 'custom-handlers'.

Patch contributed by: zeinwali@gmail.com

BUG=
Review URL: http://codereview.chromium.org/11190025/

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@500 164481a5-15cb-f69f-4b93-856c5b7754c5

21 months agoInclude 'params' url component in urlparse.
tonyg@google.com [Mon, 22 Oct 2012 19:16:54 +0000 (19:16 +0000)]
Include 'params' url component in urlparse.
Some urls are only differentiated by the 'params' section of the url, while all recorded to same resource in the archive (overwritten).

Some of these urls are encountered on yahoo.com, cnn.com and espn.com
*http://www.yahoo.com/p.gif;_ylp=A2KLf_XtrIRQshgA7SKbvZx4?t=1204251968254&hpset=0
*http://ad.doubleclick.net/adj/espn.us.com.espn/espnfrontpage;pgtyp=espnfrontpage;sp=espn;pos=InContent;u=swid=3AF068CA-4A03-42B2-8867-A4E9E5B830ED%7Cpgtyp=espnfrontpage%7Csp=espn;dcopt=ist;sz=300x600,300x250,1x1;tile=3;ord=4619999999

On Espn.com site, such urls are requested multiple times, but identified as repeated requests and recorded to the same resource. This is also an issue, while recording from different clients that might pass different arguments in the params component.

Patch contributed by: zeinwali@gmail.com

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@499 164481a5-15cb-f69f-4b93-856c5b7754c5

21 months agoMinify the deterministic script to overcome js syntax errors when injected to quoted...
tonyg@google.com [Fri, 19 Oct 2012 15:33:07 +0000 (15:33 +0000)]
Minify the deterministic script to overcome js syntax errors when injected to quoted HTML tags.
The issue is encountered in a resource on msn.com. Namely: (http://rad.msn.com/ADSAdClient31.dll?GetSAd=&DPJS=4&PN=MSFT&ID=38035C8940D1629D10A75F4B43D16259&MUID=38035C8940D1629D10A75F4B43D16259&PG=MSNIF1&AP=1455) the resource is a script which is transferred with the wrong content-type (text/html) and has a string containing "<head>" section that WPR attempts to inject the deterministic.js script to. After the injection the line breaks in deterministic.js cause a syntax error in the resulting js and blocks execution (and potentially more requests).

*This could partially solve the issue: http://code.google.com/p/web-page-replay/issues/detail?id=5
*This also works with the fix in issue: http://codereview.chromium.org/10411060/#ps5001

The msn script in question has content body that contains the following piece of code:
{var s = '<html><head><title>Advertisement</title></head><body id="'
....

Patch contributed by: zeinwali@gmail.com

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@498 164481a5-15cb-f69f-4b93-856c5b7754c5

21 months agoFix Date's prototype in deterministic.js.
tonyg@google.com [Thu, 18 Oct 2012 23:43:31 +0000 (23:43 +0000)]
Fix Date's prototype in deterministic.js.

This makes it so 'new Date() instanceof Date' properly returns true. This was
necessary for replaying Google Calendar.
Review URL: https://codereview.chromium.org/11194072

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@497 164481a5-15cb-f69f-4b93-856c5b7754c5

21 months agoadding a command to merge multiple archives (chaining resource from archives in order)
tonyg@google.com [Fri, 12 Oct 2012 15:04:49 +0000 (15:04 +0000)]
adding a command to merge multiple archives (chaining resource from archives in order)

Contributed by: zeinwali@gmail.com

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@496 164481a5-15cb-f69f-4b93-856c5b7754c5

21 months agoDon't prevent python from exiting when there is HTTP server thread activity.
tonyg@google.com [Thu, 11 Oct 2012 22:41:56 +0000 (22:41 +0000)]
Don't prevent python from exiting when there is HTTP server thread activity.

This is useful so that Chrome's webpagereplay.py wrapper doesn't hit a 5 min
timeout when exiting cleanly.
Review URL: https://codereview.chromium.org/11048035

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@495 164481a5-15cb-f69f-4b93-856c5b7754c5

23 months agoFix sysctl kwarg bug.
slamm@google.com [Wed, 29 Aug 2012 17:37:47 +0000 (17:37 +0000)]
Fix sysctl kwarg bug.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@494 164481a5-15cb-f69f-4b93-856c5b7754c5

23 months agoOnly shape dns port when running dns server.
slamm@google.com [Wed, 29 Aug 2012 17:37:41 +0000 (17:37 +0000)]
Only shape dns port when running dns server.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@493 164481a5-15cb-f69f-4b93-856c5b7754c5

23 months agoAdd sudo for commands that need it.
slamm@google.com [Wed, 22 Aug 2012 20:15:45 +0000 (20:15 +0000)]
Add sudo for commands that need it.
(Can add those commands to sudoers instead of running whole script as root.)

BUG=

Review URL: https://chromiumcodereview.appspot.com/10831417

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@492 164481a5-15cb-f69f-4b93-856c5b7754c5

23 months agoSupport HTTPS on perftracker.
simonjam@chromium.org [Sat, 11 Aug 2012 00:57:20 +0000 (00:57 +0000)]
Support HTTPS on perftracker.

BUG=none

Review URL: https://chromiumcodereview.appspot.com/10826104

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@491 164481a5-15cb-f69f-4b93-856c5b7754c5

23 months agoAdd --append option to replay.py.
slamm@google.com [Tue, 7 Aug 2012 17:51:22 +0000 (17:51 +0000)]
Add --append option to replay.py.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10829202

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@490 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoProblems of multiple headers with the same name in an http response.
fdeng@chromium.org [Tue, 24 Jul 2012 20:30:52 +0000 (20:30 +0000)]
Problems of multiple headers with the same name in an http response.

This CL is to make WPR handle multiple headers with the same name and fix send_error in
httpproxy.py.

The http response may contain multiple headers with the same name. For example,
there might be multiple "set-cookie" headers.
The correct format of these headers in the response is as bellow:

    set-cookie: GMAIL_IMP=EXPIRED; Expires=Wed, 11-Jul-2012 21:53:17 GMT;
Path=/mail; Secure
    set-cookie: GMAIL_STAT_3d81=EXPIRED; Expires=Wed, 11-Jul-2012 21:53:17 GMT;
Path=/mail; Secure

where each cookie got set by one "set-cookie" header.

However, it is observed from the client(browser) side that, multile "set-cookie"
headers becomes one line in the response from WPR, which looks like:
    set-cookie: GMAIL_IMP=EXPIRED; Expires=Wed, 11-Jul-2012 21:53:17 GMT;
Path=/mail; Secure, GMAIL_STAT_3d81=EXPIRED; Expires=Wed, 11-Jul-2012 21:53:17
GMT; Path=/mail; Secure

This bad formatting makes the client (browser) not be able to parse the
set-cookie headers properly.

The cause of this problem can be traced from httpclient.RealHttpFetch.__call__,
where it constructs the ArchivedHttpResponse using response.getheaders().

 archived_http_response = httparchive.ArchivedHttpResponse(
            response.version,
            response.status,
            response.reason,
            response.getheaders(),
            chunks,
            delays)

By looking into the value of response.getheaders, I found that the "set-cookie"
headers are formatted wrongly. I think this is a glitch in
httplib.HttpResponse.
(This issue is reported at http://bugs.python.org/issue1660009 and
http://stackoverflow.com/questions/1649401/how-to-handle-multiple-set-cookie-...)

The workaround is not to use response.getheaders() but use response.msg.headers
instead. I modified the code as such and I am able to log into gmail and other
website which requires logging in.

Another modification I made is in httpproxy.HttpArchiveHandler.send_error(self,
status). This function override the one in
BaseHTTPServer.BaseHTTPRequestHandler.
However, the signature of the function is not compatible with the one in its
parent class. This is problematic because other functions in the parent class
still make calls to this fucntion.
For example, BaseHTTPServer.BaseHTTPRequestHandler.parse_request makes call to
it via "self.send_error(400, "Bad request version (%r)" % version)".
So I believe we need to make the function signature look the same as it is in
its parent class.

BUG=None
TEST= httpclient_test.py
Review URL: https://chromiumcodereview.appspot.com/10807016

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@489 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoadd _check_output
simonjam@chromium.org [Mon, 23 Jul 2012 23:02:10 +0000 (23:02 +0000)]
add _check_output

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@488 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agouse subprocess
simonjam@chromium.org [Mon, 23 Jul 2012 23:02:08 +0000 (23:02 +0000)]
use subprocess

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@487 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoadd comment
simonjam@chromium.org [Mon, 23 Jul 2012 23:02:05 +0000 (23:02 +0000)]
add comment

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@486 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoimplement _get_cwnd
simonjam@chromium.org [Mon, 23 Jul 2012 23:02:03 +0000 (23:02 +0000)]
implement _get_cwnd

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@485 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agomore fixes
simonjam@chromium.org [Mon, 23 Jul 2012 23:01:58 +0000 (23:01 +0000)]
more fixes

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@484 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoRevive perftracker
simonjam@chromium.org [Fri, 20 Jul 2012 18:26:16 +0000 (18:26 +0000)]
Revive perftracker

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@483 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFixes after refactor in r479:
sullivan@chromium.org [Mon, 16 Jul 2012 18:48:21 +0000 (18:48 +0000)]
Fixes after refactor in r479:
Fixed log line which became inaccurate in platformsettings.py.
Fixed invalid reference to get_platform_settings in httparchive.py.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@482 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoTemporarily disable server RTT measurement to get recording working reliably.
tonyg@google.com [Thu, 12 Jul 2012 21:11:54 +0000 (21:11 +0000)]
Temporarily disable server RTT measurement to get recording working reliably.
Review URL: https://chromiumcodereview.appspot.com/10701163

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@481 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoAdd proxyshaper rate limiter tests.
slamm@google.com [Fri, 29 Jun 2012 17:09:56 +0000 (17:09 +0000)]
Add proxyshaper rate limiter tests.
Fix test-revealed issues.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10695036

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@480 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoRefactor platformsettings.
slamm@google.com [Thu, 28 Jun 2012 23:29:45 +0000 (23:29 +0000)]
Refactor platformsettings.
 - Automatically create an instance and add functions to module-level.
   (The random module uses this pattern.)
 - Use atexit.register to automate restoring values.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10542042

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@479 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoTraffic shaping in Python.
slamm@google.com [Thu, 28 Jun 2012 20:30:13 +0000 (20:30 +0000)]
Traffic shaping in Python.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10380040

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@478 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoCreate 1.1.2 for r476 release.
tonyg@google.com [Thu, 14 Jun 2012 19:44:49 +0000 (19:44 +0000)]
Create 1.1.2 for r476 release.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@477 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoProvide a fallback if pkg_resources module is unavailable.
slamm@google.com [Wed, 13 Jun 2012 16:06:48 +0000 (16:06 +0000)]
Provide a fallback if pkg_resources module is unavailable.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10546134

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@476 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoMove timer function to platformsettings.
slamm@google.com [Tue, 5 Jun 2012 15:31:18 +0000 (15:31 +0000)]
Move timer function to platformsettings.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10483021

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@475 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFix pkg_resources use.
slamm@google.com [Mon, 4 Jun 2012 23:29:11 +0000 (23:29 +0000)]
Fix pkg_resources use.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@474 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFix line endings.
slamm@google.com [Mon, 4 Jun 2012 23:26:42 +0000 (23:26 +0000)]
Fix line endings.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@473 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFor SPDY, use the generated certfile that contains the private key.
slamm@google.com [Tue, 29 May 2012 20:11:21 +0000 (20:11 +0000)]
For SPDY, use the generated certfile that contains the private key.

Make --spdy a boolean flag and rely on --no-ssl to turn off ssl.
Drop "-s" for --spdy. The former does not really save much typing.

BUG=25
TEST=

Review URL: https://chromiumcodereview.appspot.com/10453042

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@472 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoUse 100 queue slots for all platforms.
slamm@google.com [Tue, 29 May 2012 20:10:32 +0000 (20:10 +0000)]
Use 100 queue slots for all platforms.

BUG=25
TEST=

Review URL: https://chromiumcodereview.appspot.com/10443038

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@471 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoAdd timeout to pings.
slamm@google.com [Fri, 25 May 2012 16:52:50 +0000 (16:52 +0000)]
Add timeout to pings.
Check for working ping command (OsX Lion has added restrictions.)

BUG=25
TEST=

Review URL: https://chromiumcodereview.appspot.com/10451032

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@470 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoUpdate httparchive_test for create_response API change.
slamm@google.com [Wed, 23 May 2012 21:43:19 +0000 (21:43 +0000)]
Update httparchive_test for create_response API change.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10436003

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@469 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoContent length does not apply to chunked results.
tonyg@google.com [Tue, 22 May 2012 00:00:25 +0000 (00:00 +0000)]
Content length does not apply to chunked results.

Review URL: http://codereview.chromium.org/10392190

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@468 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFix deterministic script injection.
tonyg@google.com [Mon, 21 May 2012 23:22:05 +0000 (23:22 +0000)]
Fix deterministic script injection.

1. Only inject within the first 256 chars of the response. msn.com has a script which is
transferred with the wrong content-type (text/html) and has a string containing "<html>".
This patch avoids incorrectly injecting in that case.
2. Update content-length header after injection. Previously, the content was truncated
by the length of the injected script. This was causing www.youtube.com, espn.go.com and
possibly other sites to hang indefinitely.

Review URL: http://codereview.chromium.org/10411060

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@467 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFor "httparchive.py cat ...", label the parts, and add "[binary data]",
slamm@google.com [Wed, 9 May 2012 22:50:22 +0000 (22:50 +0000)]
For "httparchive.py cat ...", label the parts, and add "[binary data]",
to show response body it skipped.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10209006

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@466 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoUpdate setup.py docstring instructions.
slamm@google.com [Wed, 9 May 2012 21:22:53 +0000 (21:22 +0000)]
Update setup.py docstring instructions.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@465 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoMake get_primary_dns() work on both Snow Leopard *and* Lion.
slamm@google.com [Wed, 9 May 2012 17:44:20 +0000 (17:44 +0000)]
Make get_primary_dns() work on both Snow Leopard *and* Lion.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10389032

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@464 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoMove complexity of context manager into servermanager.
slamm@google.com [Mon, 7 May 2012 22:31:19 +0000 (22:31 +0000)]
Move complexity of context manager into servermanager.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10375002

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@463 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoPut nbhttp license in first 60 lines so it is recognized by Chrome's
slamm@google.com [Mon, 7 May 2012 21:37:30 +0000 (21:37 +0000)]
Put nbhttp license in first 60 lines so it is recognized by Chrome's
licensecheck.pl script.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10386015

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@462 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoRefer to correct license: ISC.
slamm@google.com [Mon, 7 May 2012 20:19:07 +0000 (20:19 +0000)]
Refer to correct license: ISC.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@461 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoBump easy_install package to version 1.1. Pull in fixes to --record and proper shutdown.
slamm@google.com [Thu, 3 May 2012 23:42:13 +0000 (23:42 +0000)]
Bump easy_install package to version 1.1. Pull in fixes to --record and proper shutdown.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@460 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoDocstring tweak.
slamm@google.com [Thu, 3 May 2012 23:06:28 +0000 (23:06 +0000)]
Docstring tweak.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@459 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoQuick fix for contextmanger items. Need try-finally. Otherwise, an exception will...
slamm@google.com [Thu, 3 May 2012 22:39:57 +0000 (22:39 +0000)]
Quick fix for contextmanger items. Need try-finally. Otherwise, an exception will cause code after yield to get skipped (not what I had expected).

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@458 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFix errors in record mode introduced by my recent changes.
slamm@google.com [Thu, 3 May 2012 22:22:00 +0000 (22:22 +0000)]
Fix errors in record mode introduced by my recent changes.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@457 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoAdd license boilerplate to ipaddr/test-2to3.sh.
slamm@google.com [Mon, 30 Apr 2012 20:47:14 +0000 (20:47 +0000)]
Add license boilerplate to ipaddr/test-2to3.sh.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@456 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoAdd license
slamm@google.com [Mon, 30 Apr 2012 19:42:48 +0000 (19:42 +0000)]
Add license

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@455 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoAdd ipaddr sources to WPR tree (instead of using svn:externals).
slamm@google.com [Fri, 27 Apr 2012 17:42:28 +0000 (17:42 +0000)]
Add ipaddr sources to WPR tree (instead of using svn:externals).
This will make it easier to include WPR source in other projects like Chromium.
Review URL: https://chromiumcodereview.appspot.com/10180010

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@454 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoRemove svn:executable property on files with no main method.
tonyg@google.com [Wed, 25 Apr 2012 10:16:13 +0000 (10:16 +0000)]
Remove svn:executable property on files with no main method.
Review URL: https://chromiumcodereview.appspot.com/10166031

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@453 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFixes for Python egg.
slamm@google.com [Tue, 24 Apr 2012 22:38:38 +0000 (22:38 +0000)]
Fixes for Python egg.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10201003

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@452 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoServer delay changes:
slamm@google.com [Tue, 24 Apr 2012 18:37:41 +0000 (18:37 +0000)]
Server delay changes:
* Move delay placement:
  - was: delay_0, headers, chunk_1, delay_1, ..., chunk_n, delay_n, chunk_zero_length
  - now: delay_0, headers, delay_1, chunk_1, ..., delay_n, chunk_n, chunk_zero_length
* Split ArchivedHttpResponse.server_delays into .headers_delay and .data_delays.
* Record the delay as time between chunks instead of chunk start to start.
* On record, only subtract server_rtt from first delay (had subtracted from every delay).
* On playback, make initial delay before writing headers (was at end of headers).
* Add delay info to "httparchive.py cat ...."

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10204006

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@451 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoCopy response only if inject succeeds.
slamm@google.com [Mon, 23 Apr 2012 23:11:25 +0000 (23:11 +0000)]
Copy response only if inject succeeds.
Avoid changing archive even in playback.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10161011

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@450 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFix bug in httparchive.create_response. Body needs to be a list of content.
slamm@google.com [Fri, 20 Apr 2012 23:25:18 +0000 (23:25 +0000)]
Fix bug in httparchive.create_response. Body needs to be a list of content.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@449 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoAdd import needed for r447.
slamm@google.com [Thu, 19 Apr 2012 21:09:37 +0000 (21:09 +0000)]
Add import needed for r447.

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@448 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoAdd a setup script for creating a python egg.
tonyg@google.com [Wed, 18 Apr 2012 10:17:49 +0000 (10:17 +0000)]
Add a setup script for creating a python egg.

See comment in setup.py for usage.

BUG=22
Review URL: http://codereview.chromium.org/10103033

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@447 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoFix setting up SSL.
slamm@google.com [Fri, 13 Apr 2012 13:04:52 +0000 (13:04 +0000)]
Fix setting up SSL.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/10069018

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@446 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoQuit if port already in use.
slamm@google.com [Fri, 6 Apr 2012 20:39:49 +0000 (20:39 +0000)]
Quit if port already in use.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/9969133

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@445 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoUse fixed paths for subprocess commands (to improve security).
slamm@google.com [Wed, 4 Apr 2012 17:21:11 +0000 (17:21 +0000)]
Use fixed paths for subprocess commands (to improve security).
BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/9950123

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@444 164481a5-15cb-f69f-4b93-856c5b7754c5

2 years agoClean-up temporary certificate file.
slamm@google.com [Wed, 4 Apr 2012 17:16:58 +0000 (17:16 +0000)]
Clean-up temporary certificate file.

BUG=
TEST=

Review URL: https://chromiumcodereview.appspot.com/9956122

git-svn-id: http://web-page-replay.googlecode.com/svn/trunk@443 164481a5-15cb-f69f-4b93-856c5b7754c5