webm/webmlive.git
2 years agoEoL fixes: test_shell sln/vcproj files and http_client sln file. 77/14077/2 master
Tom Finegan [Thu, 12 Jan 2012 20:48:42 +0000 (15:48 -0500)]
EoL fixes: test_shell sln/vcproj files and http_client sln file.

Convert the files to LF within the repo.

Change-Id: I982dcf1861d6ffdd0ab925f291de4f2ae9cdb6b2

2 years agoAdd baseclasses. 32/11232/5
Tom Finegan [Fri, 4 Nov 2011 19:53:26 +0000 (15:53 -0400)]
Add baseclasses.

Add the MS baseclasses source files and deps to the project.
Baseclasses sources are referenced from their installed location
within the Windows platform SDK. Currently this is set to the default
install path using http_client/environment.vsprops, which defines a
custom macro named DIRECTSHOW_BASECLASSES.

Change-Id: I672d612bb57af200d54ba20e59da2d46c5a08670

2 years agoUse CRLF EoLs in sln and vsprop files. 14/11414/1
Tom Finegan [Wed, 9 Nov 2011 21:59:45 +0000 (16:59 -0500)]
Use CRLF EoLs in sln and vsprop files.

Change-Id: I1b85173ab764afac97def7bf491874826b3a4f54

2 years agoUse CRLF in vcproj files. 31/11231/1
Tom Finegan [Fri, 4 Nov 2011 17:13:08 +0000 (13:13 -0400)]
Use CRLF in vcproj files.

Change-Id: I6f0136952524f0f846d85a26c243a475898e5936

2 years agoAdd AUTHORS, LICENSE, PATENTS files. 44/10444/1
Tom Finegan [Thu, 20 Oct 2011 21:05:11 +0000 (17:05 -0400)]
Add AUTHORS, LICENSE, PATENTS files.

All taken from libvpx.

Change-Id: I7718f5bc9f749043c49c54a30a85bac9f0331abc

2 years agoAdd support for manual a/v source configuration. 87/10287/3
Tom Finegan [Wed, 19 Oct 2011 03:53:33 +0000 (23:53 -0400)]
Add support for manual a/v source configuration.

Add manual configuration flags to base encoder interface. Implement
manual configuration support via use of directshow filter/pin property
pages.

Change-Id: Id5743d9dcea5e73031dfaaf023db25a8ac9c1397

2 years agoLint fixes. 86/10286/3
Tom Finegan [Wed, 19 Oct 2011 03:25:55 +0000 (23:25 -0400)]
Lint fixes.

- Add directory names to includes.
- Add NOLINT to std::nothrow lines. (lint false positive)
- Add NOLINT to short/long usage in basictypes.h.
- Add NOLINT to long usage in WebmBufferParser (required by libwebm parser).
- Replace non-const references with pointers.

Change-Id: I966da662fe1f61bef964c6b272e3cc5d5981f14c

2 years agoRemove debug_util and file_util. 60/10360/1
Tom Finegan [Wed, 19 Oct 2011 23:53:30 +0000 (19:53 -0400)]
Remove debug_util and file_util.

- Move HRLOG definition to where it's used: webm_encoder_dshow.
- Delete debug_util.h, file_util.h, and file_util_win.cc from project and repo.

Change-Id: Icde08bd0af33dab763c1893fe12f292f41ab4dd7

2 years agoAdd decimation. 09/9709/1
Tom Finegan [Fri, 7 Oct 2011 03:27:46 +0000 (23:27 -0400)]
Add decimation.

Change-Id: If201f83164735fbf7f9342b1e748c2c4d055183b

2 years agoAdd agent query fragment to POST target URL. 14/8914/1
Tom Finegan [Thu, 6 Oct 2011 21:31:43 +0000 (17:31 -0400)]
Add agent query fragment to POST target URL.

Change-Id: Ie6e222de66307445a99d71f26ea62ab5701e92a2

2 years agodebug_util: change HRLOG to char. 76/8676/7
Tom Finegan [Mon, 3 Oct 2011 22:54:52 +0000 (18:54 -0400)]
debug_util: change HRLOG to char.

- Remove usage of the L macro. It produces wchar_t literals,
  which do not work with glog.

Change-Id: I5b48717d38e6ea87a1c468984f75061fc110ec65

2 years agoAvoid crashing on invalid command lines. 76/8776/3
Tom Finegan [Wed, 5 Oct 2011 00:30:25 +0000 (20:30 -0400)]
Avoid crashing on invalid command lines.

Fix crash when command line ended with a valid argument
string, but was missing the argument value.

Change-Id: I8a64073f6ee4f72248a8457b17022cdf0f7466af

2 years agoAdd audio capture configuration. 73/8173/23
Tom Finegan [Mon, 3 Oct 2011 22:54:24 +0000 (18:54 -0400)]
Add audio capture configuration.

Support configuration of audio channels, sample rate,
and sample size. Note that video configuration support
code corresponding to the audio additions is included
in this commit, but is not yet in use.

Change-Id: I0b5770231e6997589b2d063c1ab63e7b5869354b

2 years agoAdd video source configuration. 72/8172/12
Tom Finegan [Thu, 22 Sep 2011 23:17:22 +0000 (19:17 -0400)]
Add video source configuration.

Add support for configuration of video source width, height, and
frame rate for webcams.

Change-Id: I48fe48713e4411f7c493a75603e1d0f61c2cae9e

2 years agoAdd video AM_MEDIA_TYPE management classes. 71/8171/4
Tom Finegan [Thu, 22 Sep 2011 22:48:36 +0000 (18:48 -0400)]
Add video AM_MEDIA_TYPE management classes.

Add MediaType and VideoMediaType for dealing with
AM_MEDIA_TYPE structures containing VIDEOINFOHEADER
and VIDEOINFOHEADER2 format blobs. Support
configuration of the header using
WebmEncoderConfig::VideoCaptureConfig structs. For
use w/video input device configuration and logging.

Also:
- disable CRT deprecation warnings at project level instead of
  via pragma
- lint fixes.

Change-Id: Ia7af4718d9bee5a01a53168d3deb4753924211fd

2 years agowebm_buffer_parser: replace ParseMode enum with func pointer. 92/7992/3
Tom Finegan [Tue, 20 Sep 2011 19:27:14 +0000 (15:27 -0400)]
webm_buffer_parser: replace ParseMode enum with func pointer.

Call function pointer directly instead of using a switch statement in
WebmBufferParser::Parse. ParseSegmentHeaders updates the pointer when
successful, eliminating the need to check the parse mode on every parsing
attempt.

Change-Id: Ic8768f0e641b3f85a1d4ef2e28102fb4d060f197

2 years agothird_party: add google glog 89/7689/6
Tom Finegan [Wed, 14 Sep 2011 01:44:03 +0000 (21:44 -0400)]
third_party: add google glog

Add glog includes and libs to third_party/glog. Replace DBGLOGs with
LOG(SEVERITY).

Change-Id: I54d1b4d8998695fd727d40621a33d157f34838a8

2 years agoAdd VP8 token partitions to CLI opts. 85/7685/1
Tom Finegan [Wed, 14 Sep 2011 00:33:16 +0000 (20:33 -0400)]
Add VP8 token partitions to CLI opts.

Also, actually implement undershoot. It was being stored, but
not set.

Change-Id: I6fa173ce7dd85b39e46e08d81405a4a8bba6a72f

2 years agoAdd vorbis audio bitrate control. 99/7599/1
Tom Finegan [Mon, 12 Sep 2011 20:47:54 +0000 (16:47 -0400)]
Add vorbis audio bitrate control.

- Add necessary includes to third_party/oggdsf.
- Add vorbis_bitrate to command line parser and usage.
- Add configuration code in webm_encoder_dshow.

Change-Id: I8b0f8927b3e5880774890e3a37316f29a8bdb70b

2 years agoRemove obsolete/unused LiveWebMStreamer. 98/7598/1
Tom Finegan [Mon, 12 Sep 2011 20:35:58 +0000 (16:35 -0400)]
Remove obsolete/unused LiveWebMStreamer.

Change-Id: Ia2a8218097223e8850ac6b10970c2a4ebb64e032

2 years agowebm_encoder_dshow: add a/v source configuration stubs. 97/7597/1
Tom Finegan [Thu, 8 Sep 2011 20:00:01 +0000 (16:00 -0400)]
webm_encoder_dshow: add a/v source configuration stubs.

Add methods ConfigureAudioSource and ConfigureVideoSource to
WebmEncoderImpl.  These do nothing outside of producing logging
output and returning kSuccess.

Change-Id: I6499903f60f61b4bc98c11b97b74967b2202f6c4

2 years agoExpand vpx configuration options.
Tom Finegan [Tue, 30 Aug 2011 19:43:59 +0000 (15:43 -0400)]
Expand vpx configuration options.

Allow for the following to be set via command line:
- bitrate
- quantizer min/max
- speed (cpu used)
- static threshold
- bitrate undershoot percentage
- thread count

Change-Id: I22d5bd24ba283fcd37376ef8208e3a9f18f80430

2 years agoSupport multipart/form and simple POSTs in the test server.
Tom Finegan [Tue, 30 Aug 2011 01:32:54 +0000 (21:32 -0400)]
Support multipart/form and simple POSTs in the test server.

Change-Id: Ib5cdc5c0d960716115ee9a06dffcf6e261241d75

2 years agoSend HTTP POSTs with WebM chunks as content-data.
Tom Finegan [Tue, 30 Aug 2011 01:31:46 +0000 (21:31 -0400)]
Send HTTP POSTs with WebM chunks as content-data.

Instead of wrapping the WebM chunk in a form, send the WebM chunk
as the POST content-data. Make the new POST method the default, and
continue to support form upload via --form_post on the command line opt.

Change-Id: I1e4c9c8f64766da14b33bf7acb1ae0294c561f8a

2 years agohttp_uploader: ensure target URL is non-empty
Tom Finegan [Mon, 29 Aug 2011 21:46:16 +0000 (17:46 -0400)]
http_uploader: ensure target URL is non-empty

This is a bug fix. HttpUploaderImpl::UploadBuffer allows the user
to pass an empty target URL, but it did not confirm that a non-empty
URL was stored.

Change-Id: I6dcd45de9e33e7e07de9cdda121cf4e2b2f06f07

2 years agoAdd support for ingestion server query params.
Tom Finegan [Fri, 26 Aug 2011 21:39:42 +0000 (17:39 -0400)]
Add support for ingestion server query params.

Ingestion servers require query params appended to the target
of the POST request in order to handle chunked WebM upload
streams.  Add support for specifying stream name and id via
command line, and implement internals (metadata flag on first
post, and itag on all posts). Also allow for override (excluding the
metadata flag) of the query params via command line.

Change-Id: I74113436fb42831af507f6792f79626936333ea5

2 years agoAdd support for specifying a/v sources via command line.
Tom Finegan [Fri, 26 Aug 2011 21:36:17 +0000 (17:36 -0400)]
Add support for specifying a/v sources via command line.

Change-Id: I2fefe45e1bf52a3b8d182b330843b7f28b1fab30

2 years agowebm_encoder_dshow: load capture device by name.
Tom Finegan [Wed, 24 Aug 2011 22:16:34 +0000 (18:16 -0400)]
webm_encoder_dshow: load capture device by name.

CaptureSourceLoader skips empty objects, which means direct use of the
index into the map of source devices when enumerating in
CaptureSourceLoader::GetSource was unreliable.  To reliably load the device,
its name must be used, so GetSource is now overloaded.
The version taking an integer argument passes the result of GetSourceName(index)
to the new version of GetSource, which uses a std::wstring, and finds a device a
matching the friendly name contained in the string.

Change-Id: I78fd3c286ca8336d734fc0d0a2347f1733006dea

2 years agohttp_client: move all vcproj entries into a single group.
Tom Finegan [Wed, 24 Aug 2011 17:28:27 +0000 (13:28 -0400)]
http_client: move all vcproj entries into a single group.

Change-Id: I403df42a1525a4f981d85e5fbd8db265b3a7fc54

3 years agobuffer_util: use non-const iterator when erasing
Tom Finegan [Mon, 15 Aug 2011 20:47:14 +0000 (16:47 -0400)]
buffer_util: use non-const iterator when erasing

MSVC is fine with const_iterator and iterator, while compile fails
with gcc when using the former.  Use vector<uint8>::iterator instead
of vector<uint8>::const_iterator to please both compilers.

Change-Id: Ic2c67f03c3caa27ee80b7dd2396fc8ff81e17852

3 years agohttp_client: fix int64 typedef
Tom Finegan [Mon, 15 Aug 2011 20:31:27 +0000 (16:31 -0400)]
http_client: fix int64 typedef

Correct the int64 typedef for non-windows builds.

Change-Id: I4754a6d187998731031a3eddb6ae6d66d2df82da

3 years agohttp_client: Convert to LF only.
Tom Finegan [Mon, 15 Aug 2011 20:08:50 +0000 (16:08 -0400)]
http_client: Convert to LF only.

All non-third-party source files converted from CRLF to LF.

Change-Id: Icd6d1d915d5fa99d0ca5f369c8516b840ebd2f63

3 years agohttp_client: use multi-byte charset
Tom Finegan [Mon, 15 Aug 2011 20:05:19 +0000 (16:05 -0400)]
http_client: use multi-byte charset

Change from unicode to multi-byte charset to minimize portability
issues.

Change-Id: I749f914c04cbd50201af8838fbaf0fc260c99d62

3 years agohttp_client: relocate windows only source files
Tom Finegan [Fri, 12 Aug 2011 15:23:09 +0000 (11:23 -0400)]
http_client: relocate windows only source files

Move them into win subdirectory, and update project/includes
accordingly.

Change-Id: I9b4bb3e8d86c82d2ecddb2bd7766c32271334569

3 years agothird_party: update to new libwebm
Tom Finegan [Thu, 11 Aug 2011 21:48:27 +0000 (17:48 -0400)]
third_party: update to new libwebm

libwebm is now a single library instead of being split into
muxer and parser libs.

Change-Id: I66d291ae5b1d6ffde15e0b8ed42c7bc5d48d785f

3 years agodebug_util: add basic logging support for non-windows
Tom Finegan [Thu, 11 Aug 2011 14:38:22 +0000 (10:38 -0400)]
debug_util: add basic logging support for non-windows

Change-Id: Ib0b8d4d2922a60ec7ee022d84816fdbf2d99724c

3 years agoImplement keyframe/cluster interval configuration.
Tom Finegan [Wed, 3 Aug 2011 02:38:33 +0000 (22:38 -0400)]
Implement keyframe/cluster interval configuration.

The WebM mux filter and VP8 encoder have been updated such that
keyframe interval directly influences cluster duration. The external
changes that led to this are:
- The VP8 encoder now supports keyframe interval control based on time,
  which solves the problem of variable input rate from webcam sources.
- The WebM muxer now accumulates cluster data until a new keyframe is
  available.

Additional changes:

http_client_base:
- Disable MSVC deprecation/safety warnings.
- "Fixing" these warning results only in non-portable code, and leaving
  them on results in having to disable "warnings as errors" because
  simple every day standard library calls like std::copy and wcstombs
  are flagged "unsafe".

http_client_main:
- Remove usage of boost::program_options.
- Parse the command line directly, and removes the very odd-looking
  code used to parse the command line with boost.
- Remove abuse of windows error codes, and define what's needed
  locally.
- Remove unnecessary and redundant includes.

http_uploader:
- Change HTTP uploader Init argument from pointer to
  const HttpUploaderSettings&.
- Expand doc comments a bit.

third_party:
- update to latest VP8 encoder IDL.

Change-Id: I2be1381dbbd4bff89d6252d16a6776fa17078f86

3 years agoUpload complete WebM file elements.
Tom Finegan [Tue, 26 Jul 2011 20:37:43 +0000 (16:37 -0400)]
Upload complete WebM file elements.

Add WebmBufferParser, WebmBufferReader, and WebmChunkBuffer.
Together the classes are used to upload complete Webm chunks by
parsing the data read from the file created by WebmEncoder. This
allows server software to resume broken uploads-- each upload
sent by the client is usable on its own, and any partial upload
can be discarded (since we're sending whole clusters).

Chunks in this context are sections of WebM files consumable by
players. The first chunk is special and must not be dropped: it
contains the EBML header, segment info, and segment tracks
elements.  All subsequent chunks are complete clusters, and can
be discarded as needed.

Change-Id: Id778e8e8c54dfcadc15625e51c941e735ad7a7bc

3 years agoReduce noise in logging output.
Tom Finegan [Fri, 29 Jul 2011 15:37:04 +0000 (11:37 -0400)]
Reduce noise in logging output.

- file_reader_win: comment out open/close DBGLOGs.
- http_uploader: comment out progress callback DBGLOG.

Change-Id: I0945588c95e90f667df939fa0ea1985bdad94f37

3 years agothird_party: fix mkvparser.lib code generation
Tom Finegan [Wed, 27 Jul 2011 00:33:59 +0000 (20:33 -0400)]
third_party: fix mkvparser.lib code generation

- change from multithreaded DLL to multithreaded (static).

Change-Id: I6b2645bbd9ab0d85a3038246c08bf15fe7f6ac91

3 years agoFix release build.
Tom Finegan [Mon, 25 Jul 2011 22:02:48 +0000 (18:02 -0400)]
Fix release build.

Not sure where patchset 2 went from the last commit...

- Fix path to release mkvparser.lib.
- Add missing include to file_reader.h.

Change-Id: I8f40043dfe78aceec0f9d14431addcd18a880d33

3 years agothird_party: add libwebm (with LICENSE.TXT)
Tom Finegan [Mon, 25 Jul 2011 20:30:11 +0000 (16:30 -0400)]
third_party: add libwebm (with LICENSE.TXT)

This includes only the parser.

Change-Id: I74f450ebccc6750c0c083219961c508b710ab7d5

3 years agohttp_uploader: fix stats
Tom Finegan [Mon, 25 Jul 2011 19:50:23 +0000 (15:50 -0400)]
http_uploader: fix stats

HttpUploader stats have reset to 0 each time UploadBuffer was
called since the change from uploading files to uploading buffers.
This was happening because the stats were copied from the libcurl
progress values directly.  HttpUploaderStats now has values for
current number of bytes sent (for the present upload, if any), and
total number of bytes sent.

- Add curl_easy_getinfo call in uploader to obtain total size of
  upload from libcurl after curl_easy_perform finishes.
- Change progress callback upload rate calculation to include
  total_bytes_uploaded.
- Change main loop to use bytes_sent_current + total_bytes_uploaded
  for total bytes sent.

Change-Id: I74d68d78e95e7709b9632d8563dc4350d79b9fc2

3 years agoStyle fixes...
Tom Finegan [Mon, 25 Jul 2011 18:53:57 +0000 (14:53 -0400)]
Style fixes...

- Fix remaining instances of incorrectly named include guards.
- Fix remaining instances of incorrectly ordered includes.
- Remove once pragmas.
- Remove chromium basictypes.h; replace with minimal version.
- Rename WebmLive namespace to webmlive.
- Rename DISALLOW_COPY_AND_ASSIGN macro to
  WEBMLIVE_DISALLOW_COPY_AND_ASSIGN.

Change-Id: I231dbe7c9056a870c7f7d452ed8feae9821a463b

3 years agohttp_uploader: style fixes and additional comments.
Tom Finegan [Wed, 20 Jul 2011 19:16:56 +0000 (15:16 -0400)]
http_uploader: style fixes and additional comments.

Change-Id: Ic73c0d49cccdedea28c5b590323543b787a1456e

3 years agofile_reader: cpplint recommended style fixes
Tom Finegan [Wed, 20 Jul 2011 19:09:45 +0000 (15:09 -0400)]
file_reader: cpplint recommended style fixes

Change-Id: Id02da35b964c508c5bc7035cb726931a6bf3842d

3 years agodebug_util: cpplint recommended style fixes
Tom Finegan [Wed, 20 Jul 2011 19:08:08 +0000 (15:08 -0400)]
debug_util: cpplint recommended style fixes

Change-Id: I7411de9f450213bd2c4934ad61ff296c77352dc4

3 years agoMerge "webm_encoder: style fixes/documentation comments."
Tom Finegan [Mon, 25 Jul 2011 14:06:27 +0000 (07:06 -0700)]
Merge "webm_encoder: style fixes/documentation comments."

3 years agowebm_encoder: style fixes/documentation comments.
Tom Finegan [Wed, 20 Jul 2011 00:10:34 +0000 (20:10 -0400)]
webm_encoder: style fixes/documentation comments.

Fix style violations and add comments to the encoder source files.

Change-Id: Ib7aec69dfe94c8e9e8416974c7db5e18d8d0ae52

3 years agobuffer_util: add documentation comments.
Tom Finegan [Wed, 20 Jul 2011 19:05:21 +0000 (15:05 -0400)]
buffer_util: add documentation comments.

Change-Id: I07fbf3c834fe02e1b299f6e7f687821bcaf5cfb5

3 years agoAdd cpplint.py.
Tom Finegan [Mon, 18 Jul 2011 19:07:11 +0000 (15:07 -0400)]
Add cpplint.py.

Change-Id: I5ca9c13d98d6cf283921378b5a4d1cb0f9d52449

3 years agohttp_uploader: add comments and tidy up a bit.
Tom Finegan [Mon, 18 Jul 2011 19:06:04 +0000 (15:06 -0400)]
http_uploader: add comments and tidy up a bit.

Add comments explaining function and usage of HttpUploader and
HttpUploaderImpl.

- Remove redudant copy of HttpSettings from HttpUploader.

Change-Id: I92b1c965938623b0a8bd6ddc7adc9b5316b1c4ed

3 years agofile_reader: add comments and tidy up a bit.
Tom Finegan [Mon, 18 Jul 2011 17:21:15 +0000 (13:21 -0400)]
file_reader: add comments and tidy up a bit.

Add comments explaining function and usage of FileReader and FileReaderImpl.

FileReaderImpl:
- Use FileReader status codes to define local versions for brevity.
- Add kModeCreate and kModeOpen to status codes enum, and use those with
  Open instead of passing a pointer that gets forwarded to _wfsopen.
- Use an int instead of a wchar_t* for the mode parameter of Open.

Change-Id: I0ec3dbbf2e8df734e477cc3c1c3f30e3402058be

3 years agofile_reader: always reopen the file when attempting to read.
Tom Finegan [Sun, 17 Jul 2011 02:37:58 +0000 (22:37 -0400)]
file_reader: always reopen the file when attempting to read.

This seems to allow http_client to properly read the file as the encoder writes
it.  Read now Open's, checks available data via GetBytesAvailable, and then
reads available data into the user buffer.

- Add open mode argument to FileReaderImpl::Open to allow for reopens,
  and always close the file pointer if non-null in Open.
- Make FileReaderImpl::GetBytesAvailable private.
- Remove GetBytesAvailable from the public FileReader interface.

Change-Id: I489bb5bf0e8107d40c8c3cfb4c01de0462374a0a

3 years agohttp_client: Move file i/o outside of the uploader.
Tom Finegan [Tue, 12 Jul 2011 19:11:04 +0000 (15:11 -0400)]
http_client: Move file i/o outside of the uploader.

Note: This works, but there's an odd bug with a TODO in http_client_main.
For some reason FileReader can read data only if I open an explorer window
in the directory of the output file.  If that's done everything works well.
If not, no uploading actually takes place.  Landing this anyway-- I know this
can be solved with some focused work on the reader, which will be in the next
commit.

Many, many changes in this commit.

file_reader/file_reader_win:
- Rename Init to CreateFile
- Use FILE* instead of ifstream.
- Use _wfsopen, and open the file in w+b mode (allow reading and writing),
  with _SH_DENYNO flag (so the encoder can write to the file).

http_client:
- Add new files buffer_util.cc and buffer_util.h, which define
  WebmLive::LockableBuffer.

http_client_main:
- Add FileReader to the main application loop, and initialize it before
  anything else.
- Reorganize main into simple function calls.
- Pass data read via HttpUploader::UploadBuffer instead of reading the
  file in the uploader.  In the future this will allow for using encoder state
  to determine when to upload data instead of blindly reading as much
  data as possible.
- Add 100ms sleep per iteration of the main loop to avoid hammering on
  the filesystem once the upload catches up to end of the output file.

HttpUploader:
- Remove the Upload method, and replace with UploadBuffer and
  UploadComplete.
- Add TODO for implementing UploadFile for upload of existing files.

HttpUploaderImpl:
- Add boost::condition_variable for waking UploadThread when users call
  UploadBuffer.
- Add LockableBuffer to store user data during upload.  It remains locked for
  the duration of the upload, which stops users from overwriting or clearing the
  buffer while UploadThread is using it.
- Add UploadComplete and a bool flag to allow user code to check for upload
  completion.
- Copy uploader settings passed to Init.
- Remove hard coded file name from buffer upload form (use real local name).
- Remove Final method, and clean up in dtor instead.
- Style fixes.

webmstreamserver.py:
- revert to apending all posted data to single file

Change-Id: Ie5af8cff42d4fe4012451258c783737ddb583ceb

3 years agohttp_uploader: begin FileReader removal
Tom Finegan [Tue, 12 Jul 2011 01:49:57 +0000 (21:49 -0400)]
http_uploader: begin FileReader removal

- Refactor in progress; http_client is broken!  It compiles, but it won't upload
  anything.
- Remove ReadCallback, and add SetUploadBuffer. This passes a
  buffer pointer directly into libcurl, which it will then use for the upload.
- Rename SetupForm to SetUserFormVariables, and remove file form setup
  so it behaves as advertised.
- Remove upload pause support.
- Rename Upload to UploadBuffer.
- Remove FileReader member from HttpUploaderImpl.
- Add filename constant, kFileName.  This is the file name the HTTP server sees
  in the form when using libcurl to upload a buffer.

Change-Id: Ieb2cecb6ffea96e1f67ce11ffa4b9f64c3e1ea96

3 years agohttp_uploader: clean up HttpUploaderImpl declaration order
Tom Finegan [Tue, 12 Jul 2011 00:17:40 +0000 (20:17 -0400)]
http_uploader: clean up HttpUploaderImpl declaration order

- Add ptr_form_end_, and store the last form item pointer from libcurl.

Change-Id: I1d6c2d847fa4d969d2cdc292016e4ccd41147ec0

3 years agoFileReader: add start offset argument to Init
Tom Finegan [Mon, 11 Jul 2011 23:57:14 +0000 (19:57 -0400)]
FileReader: add start offset argument to Init

- Add Init overload with same to FileReaderImpl, which sets bytes_read_ and
  calls Init.  File reads all go through OpenAtReadOffset, so this takes care of
  the implementation details as well.
- Move all reader return codes into public FileReader interface.

Change-Id: Iaa6428b229e9706e62a5b831daf346a2b85b6415

3 years agohttp_uploader: fix curl clean up
Tom Finegan [Mon, 11 Jul 2011 18:58:37 +0000 (14:58 -0400)]
http_uploader: fix curl clean up

- fix memory leak: free the form and header lists

Change-Id: I546d4d9e3267f3c1d864da91c812509aa530f4bc

3 years agofile_reader_win: do not store return value from Open
Tom Finegan [Mon, 11 Jul 2011 18:57:22 +0000 (14:57 -0400)]
file_reader_win: do not store return value from Open

Change-Id: I331b3897522df859afc5cbb8605ea1e301f479b5

3 years agofile_reader_win: close the file between reads
Tom Finegan [Fri, 1 Jul 2011 02:24:42 +0000 (22:24 -0400)]
file_reader_win: close the file between reads

Change-Id: I32fcb903f61e967fa6384b49ed74817de4f3af8e

3 years agohttp_uploader: Add pause/resume support.
Tom Finegan [Thu, 30 Jun 2011 07:36:27 +0000 (03:36 -0400)]
http_uploader: Add pause/resume support.

- Note: the resume part doesn't work. Afaict I'm using the libcurl API right, but
  after resuming the transfer I always get a CURLE_OK from curl_easy_perform,
  which isn't expected at all, and a 301 from the test server (normal).

Change-Id: I9b885eb88b0b7e0913a3f24168c1d100cb510a49

3 years agowebm_encoder_dshow: make filter name constants static
Tom Finegan [Thu, 30 Jun 2011 07:33:35 +0000 (03:33 -0400)]
webm_encoder_dshow: make filter name constants static

Change-Id: I70e9283bb9f338d884706999927054607d906549

3 years agofile_util: add stat based implementation of get_file_size
Tom Finegan [Thu, 30 Jun 2011 07:32:54 +0000 (03:32 -0400)]
file_util: add stat based implementation of get_file_size

- Keep GetFileAttributesEx version around.  For now, at least.

Change-Id: Ia84985db66d7113a457f7d562495ab2b8c22fa4b

3 years agohttp_client: add live WebM encoding.
Tom Finegan [Thu, 30 Jun 2011 06:34:38 +0000 (02:34 -0400)]
http_client: add live WebM encoding.

- Run the WebmEncoder.
- Add duration accessor to WebmEncoder, GetEncodedDuration.
- Add encoded duration to progress output.

As a temporary measure until upload pausing is implemented, the client waits
until the encoder reports duration > 5 seconds before starting the uploader.

Change-Id: I99075456757e11179f069b2745e8743aa9711808

3 years agowebm_encoder_dshow: complete initial implementation
Tom Finegan [Thu, 30 Jun 2011 06:30:18 +0000 (02:30 -0400)]
webm_encoder_dshow: complete initial implementation

- Add final graph setup and run code.
- Add thread safe encoded duration storage/access.
- Implement encoder thread (monitors graph, keeps duration updated).

Change-Id: I60b32f28c15cd6e5e09fea1c0efb3a134be6f087

3 years agohttp_uploader: fix dead lock
Tom Finegan [Thu, 30 Jun 2011 05:49:15 +0000 (01:49 -0400)]
http_uploader: fix dead lock

- Stop was calling release() on the mutex.  unlock() must be called.

Change-Id: Ib7f99f933c7d728ac8b2f1a8a71aeb92f3efd74e

3 years agofile_reader_win: use std::ifstream for i/o
Tom Finegan [Thu, 30 Jun 2011 05:48:29 +0000 (01:48 -0400)]
file_reader_win: use std::ifstream for i/o

- std::ifstream can open files created/in-use by the DirectShow file
  writer filter.

Change-Id: I95a9c5c21c218d4ce48b73bef8e6576d6dc20622

3 years agohttp_uploader: define status codes
Tom Finegan [Thu, 30 Jun 2011 02:27:19 +0000 (22:27 -0400)]
http_uploader: define status codes

- Add status code enum to HttpUploader and HttpUploaderImpl.
- Switch from mix of COM and WINAPI error codes to our new ones.

Change-Id: I642b9445a9ac1fb708f2a4fea7e1ed87668980a1

3 years agowebm_encoder_dshow: add File Writer setup
Tom Finegan [Wed, 29 Jun 2011 21:49:53 +0000 (17:49 -0400)]
webm_encoder_dshow: add File Writer setup

- Add File Writer filter creation, configuration, and connection code.
- Add FindStreamInput/OutputPin and FindInputPin to PinFinder.
- Add IsStream to PinInfo.

Change-Id: I8011d5ea8b25619f2ebad86a9082ebf158699821

3 years agofile_reader: use FileUtil helper functions
Tom Finegan [Wed, 29 Jun 2011 20:48:17 +0000 (16:48 -0400)]
file_reader: use FileUtil helper functions

- Replace direct windows API calls with calls to our helpers.

Change-Id: Ib8aba58c1e233880042a9c6dbbc17129947265f4

3 years agohttp_client: add file_util
Tom Finegan [Wed, 29 Jun 2011 20:46:19 +0000 (16:46 -0400)]
http_client: add file_util

- Add FileUtil::file_exists and FileUtil::get_file_size to WebmLive
  namespace.

Change-Id: Idc32a9b506348dd5f4399619cdb7281cf1c8e307

3 years agowebm_encoder_dshow: add muxer setup
Tom Finegan [Wed, 29 Jun 2011 20:07:24 +0000 (16:07 -0400)]
webm_encoder_dshow: add muxer setup

- Add webm muxer filter creation, configuration, and connection code.
- Enable live mux mode.

Change-Id: Idaa10a8468f21349cbdc70645c5e9b83e17b5bb5

3 years agowebm_encoder: correct error value range
Tom Finegan [Wed, 29 Jun 2011 20:06:10 +0000 (16:06 -0400)]
webm_encoder: correct error value range

- The skipped of values were unintentional.

Change-Id: Iec7bfb171bf0b3a4cd2263fe00fb2edb66691e68

3 years agowebm_encoder_dshow: add basic Vorbis encoder setup
Tom Finegan [Wed, 29 Jun 2011 19:04:55 +0000 (15:04 -0400)]
webm_encoder_dshow: add basic Vorbis encoder setup

- Add Vorbis encoder filter creation, configuration, and connection code.
- Note: no configuration done.  Using vorbis encoder filter defaults.

Change-Id: Ic10723bf38c931e39d9b1245ffac65b65bfd5851

3 years agowebm_encoder_dshow: fix map indexes in CaptureSourceLoader
Tom Finegan [Wed, 29 Jun 2011 18:24:49 +0000 (14:24 -0400)]
webm_encoder_dshow: fix map indexes in CaptureSourceLoader

The map of capture sources filled by CaptureSourceLoader::FindAllSources
is intended to be used with 0 based consecutive indexes.  FindAllSources was
incrementing the index value used to assign capture device name when
skipping devices, which would potentially result in a map with names at, for
example, indexes 1, 3 and 6, with 0, 2, 4, and 5 empty.

Change-Id: Ib2693cecb9e8130b2a70f8da458b0d8d6d34b8af

3 years agowebm_encoder_dshow: add basic VP8 encoder setup
Tom Finegan [Wed, 29 Jun 2011 06:05:36 +0000 (02:05 -0400)]
webm_encoder_dshow: add basic VP8 encoder setup

- Add VP8 encoder filter creation, configuration, and connection code.
- Correct value of CLSID_VP8Encoder (was set to decoder CLSID).
- Fix leaking PinFinder::Find... methods.
- Note: only works if video source can connect directly to VP8 encoder. TODO
  added about use of WebM color conversion filter when direct connection
  fails.

Change-Id: Ife2c02c854d43ec4eb03b9c94e423dc8453f851e

3 years agohttp_client: initial commit of DShow encoder
Tom Finegan [Wed, 29 Jun 2011 04:44:09 +0000 (00:44 -0400)]
http_client: initial commit of DShow encoder

Add partial implementation of WebM DShow encoder.

WebmLive::WebmEncoder is the basic public interface defined in
the webm_encoder source files.  It uses WebmEncoderImpl, which
is a directshow encoder on Windows and is defined in the
webm_encoder_dshow source files.

Notes:
- Dead simple uploader functionality still works.
WebmEncoderImpl notes:
- Uses live sources only at present.
- Uses first available source of each type, audio and video.
- Uses default format for each device.
- Goes no further than creating graph and adding sources.
- Smart enough to use audio output pin when the video source
  exposes one.
- Lot's of TODO's related to above.

Change-Id: I45a0709b1090b07d22cfa5565fb540850e2f7df4

3 years agodebug_util: write clickable messages in DBGLOG's
Tom Finegan [Wed, 29 Jun 2011 04:35:09 +0000 (00:35 -0400)]
debug_util: write clickable messages in DBGLOG's

- Add ability to jump to DBGLOG location upon clicking in the
  visual studio output console.

Change-Id: I12027209d954d6f005aa61a2881ad071539f4057

3 years agohttp_uploader: simplify public interface
Tom Finegan [Tue, 28 Jun 2011 21:52:34 +0000 (17:52 -0400)]
http_uploader: simplify public interface

- Move threading stuff into HttpUploaderImpl, and hide it from
  the user.
- Rename HttpUploader/Impl::Go to Run
- Change Run and Stop to return int instead of void

Change-Id: I202ef4cc96a18ea2a2aa9059857715ab70592330

3 years agohttp_client: fix stats
Tom Finegan [Mon, 27 Jun 2011 22:26:44 +0000 (18:26 -0400)]
http_client: fix stats

- use correct variable in curl progress callback
- also tweak console output slightly

Change-Id: I95fe7211b065c0ef47ffaa0d9f89b7ed156ad87b

3 years agofile i/o refactor
Tom Finegan [Mon, 27 Jun 2011 22:25:33 +0000 (18:25 -0400)]
file i/o refactor

- again (last time, hopefully)
- move to windows API (HANDLE/OpenFile/ReadFile etc) file i/o
- move remaining windows specific stuff from FileReader to FileReaderImpl
- add enum with reader return codes to FileReader, and update FileReaderImpl
  accordingly
- move FileReader to file_reader.cc (new file)
- move FileReaderImpl declaration to file_reader_win.h (new file)
- add above new files to http_client project

Change-Id: I6d5b0d29bc1104bf6cb78bf1a8921cb8e0a03bf3

3 years agowebmstreamserver: fix file writing
Tom Finegan [Mon, 27 Jun 2011 22:19:55 +0000 (18:19 -0400)]
webmstreamserver: fix file writing

- generate new file name for each upload
- log file name to console
- open files in write instead of append mode

Change-Id: I255a19c54dbbb41189d675fad410dec8638a56c4

3 years agohttp_uploader: proper stop support
Tom Finegan [Thu, 23 Jun 2011 00:00:15 +0000 (20:00 -0400)]
http_uploader: proper stop support

- move stop flag into HttpUploaderImpl
- remove volatile bool hack; we now obtain the uploader's mutex
  before reading or changing the stop state
- make callback methods respect the stop flag

Change-Id: Ia64daa5b48b22539819ecbdb6bb870ccd4dde52a

3 years agohttp_client: add progress output
Tom Finegan [Wed, 22 Jun 2011 23:55:21 +0000 (19:55 -0400)]
http_client: add progress output

- also fix bytes per second calculation in
  HttpUploaderImpl::ProgressCallback

Change-Id: I905136d7f1c9a8218b44f71de967323b8e70dafb

3 years agohttp_uploader: add basic stats
Tom Finegan [Tue, 21 Jun 2011 20:54:28 +0000 (16:54 -0400)]
http_uploader: add basic stats

- allow user to obtain bytes uploaded and average upload rate

Change-Id: I1b13651bda1ddf927193e933d2c2db95988bf72b

3 years agowebmstreamserver: fix bug in test script
Tom Finegan [Tue, 21 Jun 2011 00:36:45 +0000 (20:36 -0400)]
webmstreamserver: fix bug in test script

- Write response data after response code

Change-Id: I95532a1f90eb9b127bdcf35ccf476a101e5278da

3 years agofile_reader: hide file i/o implementation
Tom Finegan [Tue, 21 Jun 2011 00:36:08 +0000 (20:36 -0400)]
file_reader: hide file i/o implementation

- Use scoped_ptr wrapped FileReaderImpl to hide all platform specific
  implementation details.
- Move file open/read and running byte total into FileReaderImpl.

Change-Id: I412f694025909c8610ad0d7709d5d791e676261b

3 years agohttp_uploader: refactor/cleanup and a bug fix
Tom Finegan [Thu, 16 Jun 2011 19:42:47 +0000 (15:42 -0400)]
http_uploader: refactor/cleanup and a bug fix

- bug fix: remove curl_easy_setopt that was enabling CURLOPT_UPLOAD.
  Setting CURLOPT_UPLOAD sends a HTTP PUT, and we want to send a POST.
- Move callback setup code into HttpUploaderImpl::SetCurlCallbacks
- Add LOG_CURL_ERR and LOG_CURLFORM_ERR macros (for brevity in curl
  setup code)

Change-Id: I8db90505e664696b663baac7bc4e3769fcfe62ad

3 years agotesting: relocate HTTPServer based test script
Tom Finegan [Thu, 16 Jun 2011 19:40:04 +0000 (15:40 -0400)]
testing: relocate HTTPServer based test script

Change-Id: Ied2c6df107442d015622691cc6a4bfacf2abdde7

3 years agotesting: add uploader and simple server config
Tom Finegan [Thu, 16 Jun 2011 02:42:02 +0000 (22:42 -0400)]
testing: add uploader and simple server config

- add python script for handling form upload
- add lighttpd.conf

Change-Id: Id376c5b84c964ed992400710e084626913bfbbee

3 years agohttp_uploader: complete initial implementation
Tom Finegan [Thu, 16 Jun 2011 02:38:07 +0000 (22:38 -0400)]
http_uploader: complete initial implementation

- known broken!
- well, it could be my test server, but I have my doubts about the
  present state of things.
- disable HTTP 100 (specify Expect:), and set user headers
- add WriteCallback for diagnostic purposes
- add curl_easy_perform call in UploadThread/we do something now(badly)
- add comments, hooray \o/

Change-Id: Id62f5571cbdb23bf107137bffb796d29ef4f5584

3 years agocurl: update to latest devel release
Tom Finegan [Thu, 16 Jun 2011 02:35:22 +0000 (22:35 -0400)]
curl: update to latest devel release

- v7.21.6; update required in order to use CURLFORM_STREAM
- origin http://curl.haxx.se/dlwiz/?type=lib&os=Win32&flav=-
- download mirror http://www.gknw.net/mirror/curl/win32/curl-7.21.6-devel-mingw32.zip
- note: has support for far more protocols than needed!
- vcproj post build copy step updated with new DLL deps
- add new dll deps to build_config_win

Change-Id: I5e0296215c0c3b9b1f8d427c4f813ff15c34f449

3 years agohttp_client: add build_config_win
Tom Finegan [Wed, 15 Jun 2011 22:59:30 +0000 (18:59 -0400)]
http_client: add build_config_win

- Add build configuration file for controlling link dependencies.
- Remove libs from vcproj settings.

Change-Id: If76d6672578b9702f7818f0a966e84da9ccbdd7d

3 years agohttp_uploader: complete curl init
Tom Finegan [Wed, 15 Jun 2011 00:17:49 +0000 (20:17 -0400)]
http_uploader: complete curl init

Add constants for form setup, and finish form initialization code.

We should be good to call curl_easy_perform now; unfortunately this won't
build against the libcurl currently in-tree because we must upgrade to curl
v7.18.2 or higher in order to use CURLFORM_STREAM.  We must use
CURLFORM_STREAM to upload the file as part of a form-- to avoid this
we would have to send HTTP PUT instead of HTTP POST with form data.

Beyond that we need:
- some heuristic for distinguishing between catching up to the file's write
  pointer and file creation completion in FileReader::Read
- code for handling situations where FileReader::Read cannot return any
  data

Change-Id: Ib63b6431fc66e87f35cb27c7073ec20da527ccc7

3 years agohttp_client_main: comment explaining header opt
Tom Finegan [Tue, 14 Jun 2011 21:25:32 +0000 (17:25 -0400)]
http_client_main: comment explaining header opt

- covers var opt as well

Change-Id: Ia5cafb0a7ed1a8f99b9d00d7a31ab4dcad395d28

3 years agofile_reader_win: fix running total
Tom Finegan [Tue, 14 Jun 2011 21:24:54 +0000 (17:24 -0400)]
file_reader_win: fix running total

- FileReader::Read was not updating bytes_read_

Change-Id: I950d39ca377ceda989a32682bcbc2daadd6d05af

3 years agohttp_client: add form variable and http header options
Tom Finegan [Tue, 14 Jun 2011 03:18:31 +0000 (23:18 -0400)]
http_client: add form variable and http header options

- http_client_main: add basic support for passing form variables and
  HTTP headers via command line
- http_uploader: Add stub FormUploadInit, which will handle sending
  form variable names and values to libcurl.  Add StringMap type to
  HttpUploaderSettings for storage of form variables and http headers.

Change-Id: I008b4f874d0d35d5ccf8d68c3a0cbcaf2d90eb97

3 years agohttp_client_main: add missing copyright header
Tom Finegan [Tue, 14 Jun 2011 03:13:41 +0000 (23:13 -0400)]
http_client_main: add missing copyright header

Change-Id: I3062f1be8b8f88cc69c402b044622cb5cd6a6f74

3 years agohttp_uploader: begin i/o implementation
Tom Finegan [Tue, 14 Jun 2011 03:12:59 +0000 (23:12 -0400)]
http_uploader: begin i/o implementation

- add FileReader to HttpUploaderImpl
- add FileReader initialization

Change-Id: If7a9776fd393d1d4f35c2b78e828a3fe67c59485