native_client/pnacl-llvm.git
9 hours agoAdd SandboxMemoryAccesses MinSFI LLVM pass master
David Brazdil [Tue, 29 Jul 2014 17:35:23 +0000 (10:35 -0700)]
Add SandboxMemoryAccesses MinSFI LLVM pass

This pass sandboxes all pointer-type arguments of memory access instructions. See the comments in lib/Transforms/MinSFI/SandboxMemoryAccesses.cpp for details.

Creates new MinSFI directories and modifies corresponding build files.

BUG= https://code.google.com/p/chromium/issues/detail?id=397177
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check)
      manually tested on zlib compiled with PNaCl

R=jvoung@chromium.org, mseaborn@chromium.org

Review URL: https://codereview.chromium.org/390003005

8 days agoPNaCl atomics: clean up some of the code
JF Bastien [Mon, 21 Jul 2014 20:18:53 +0000 (13:18 -0700)]
PNaCl atomics: clean up some of the code

One of the class' methods isn't used anymore, and the ordering of error messages
can be slightly improved by refactoring some of the code to explicitly check for
invalid overloads.

R=dschuff@chromium.org
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check)
BUG= none

Review URL: https://codereview.chromium.org/408613002

11 days agoFix pnacl-bcdis abbreviation records.
Karl Schimpf [Fri, 18 Jul 2014 22:27:30 +0000 (15:27 -0700)]
Fix pnacl-bcdis abbreviation records.

Due to a bug in NaClBitcodeParser::SetBID, the record values were not
cleared, resulting in the block ID of the SetBID being left in the
succeeding abbreviation record. This CL fixes this problem.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/401903003

2 weeks agoImprove atomic tests
JF Bastien [Tue, 15 Jul 2014 01:16:28 +0000 (18:16 -0700)]
Improve atomic tests

The testing for atomics was somewhat insufficient, and the file was getting too big. I split up the file and added more tests:
 - Use CHECK-LABEL, which we didn't have back when the tests were written.
 - Move the CHECK-NEXT for ret to the same line as the ret instruction, to make things more readable.
 - The atomic.ll file is now split up in 5 files.
 - Add the atomic_seq_cst.ll file which performs the same tests as volatile.ll but on sequentially-consistent atomics.
 - Add the atomic_others.ll file, which checks that the other atomic operations get promoted to seq_cst. This isn't as thorough as seq_cst and volatile testing because everything currently gets rewritten to seq_cst.

R=dschuff@chromium.org
TEST= self

Review URL: https://codereview.chromium.org/382243006

2 weeks agoRefactor code so that intrinsics checks can be used by pnacl-bcdis.
Karl Schimpf [Mon, 14 Jul 2014 22:26:34 +0000 (15:26 -0700)]
Refactor code so that intrinsics checks can be used by pnacl-bcdis.

Moves code about intrinsic checks into a separate file, so that
it can be used by a) the PNaCl ABI verifier; b) pnacl-bcdis, and
c) NaClBitcodeReader can use to process signatures of intrinsics.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3894
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/374703003

2 weeks agoTranslator: rematerialize constant vector loads
JF Bastien [Fri, 11 Jul 2014 23:36:58 +0000 (16:36 -0700)]
Translator: rematerialize constant vector loads

PNaCl's ABI doesn't contain constant vectors, they therefore need to be
rematerialized by the translator if we want to get the best optimizations
possible.

The CL also renames CombineVectorInstructions.cpp to BackendCanonicalize.cpp to
be more accurate about what the pass does: it can now affect more than vectors
because the above change affects bitcasts and loads in general (since it doesn't
make sense to only stick with vectors). Things should stay in this one pass so
that the translator can restrict itself to what's important to optimize, and do
so as fast as it can.

R=dschuff@chromium.org
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3893
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check)

Review URL: https://codereview.chromium.org/385073012

2 weeks agoReduce redundancy created by PNaCl's vector globalization pass
JF Bastien [Wed, 9 Jul 2014 20:31:19 +0000 (13:31 -0700)]
Reduce redundancy created by PNaCl's vector globalization pass

This change makes it easier to reconstruct shuffles, and reduces the pass'
reliance on constant merging. It's a toolchain only fix, meaning that no
translator update is needed, only a recompile.

The loads of constant vectors are still present and will be handled in a
separate change.

R=dschuff@chromium.org
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3893
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check)

Review URL: https://codereview.chromium.org/379013002

3 weeks agoTurn off type signature checking in pnacl-bcdis.
Karl Schimpf [Wed, 2 Jul 2014 18:37:36 +0000 (11:37 -0700)]
Turn off type signature checking in pnacl-bcdis.

Removes bug caused by inconsistency in PNaCl ABI document. States that
integral function parameter/return values must only be i32/i64.
However, for calls to (whitelisted) intrinsic funtions, type signatures can
have any integral size.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3894
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/362093002

3 weeks agoGlobalizeConstantVectors: set Unnamed addr
JF Bastien [Wed, 2 Jul 2014 17:52:48 +0000 (10:52 -0700)]
GlobalizeConstantVectors: set Unnamed addr

The globalized vectors can be merged by constant merge when it knows that the
address doesn't matter, only the content does.

R=dschuff@chromium.org
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check)
BUG= none

Review URL: https://codereview.chromium.org/367963006

3 weeks agoPNaClABIErrorReporter: remove last virtual
JF Bastien [Wed, 2 Jul 2014 17:38:22 +0000 (10:38 -0700)]
PNaClABIErrorReporter: remove last virtual

The previous CL:
  https://codereview.chromium.org/349683004/

Left one virtual method in PNaClABIErrorReporter.

R=kschimpf@google.com, kschimpf@chromium.org
BUG= none
TEST= none

Review URL: https://codereview.chromium.org/362173002

4 weeks agoPNaCl SIMD: don't perform constant propagation in expand-shufflevector
JF Bastien [Tue, 1 Jul 2014 16:16:22 +0000 (09:16 -0700)]
PNaCl SIMD: don't perform constant propagation in expand-shufflevector

Using the IR builder leads to extra optimizations which usually would be useful,
but in the case of shufflevector they make reconstructing the shuffles harder
when translating. This change therefore uses manual instruction insertion, and
adjusts the tests accordingly.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3893
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check)
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/365503004

4 weeks agoAdd skip block method to NaCl bitcode parser.
Karl Schimpf [Mon, 30 Jun 2014 17:55:17 +0000 (10:55 -0700)]
Add skip block method to NaCl bitcode parser.

This is needed by Subzero, to be able to skip blocks.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3890
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/357303003

4 weeks agoFix comments about PIC return address calculation
Derek Schuff [Mon, 30 Jun 2014 16:00:23 +0000 (09:00 -0700)]
Fix comments about PIC return address calculation

The previous commit failed to update them to match the code.

R=mseaborn@chromium.org
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3873

Review URL: https://codereview.chromium.org/347863007

4 weeks agoMake x86-64 sandbox-base-hiding call sequence work with PIC
Derek Schuff [Fri, 27 Jun 2014 23:16:37 +0000 (16:16 -0700)]
Make x86-64 sandbox-base-hiding call sequence work with PIC

The call sequence used on x86-64 to avoid storing the sandbox base
address on the stack was not PIC-friendly because it pushed the return
address as an immediate. When generating PIC code, manually calculate
the return address into a register and push that.

This includes a tiny cherry-pick of upstream r202418, which adds an
accessor for the relocation model to MCObjectFileInfo

R=mseaborn@chromium.org
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3873
TEST= LLVM regression

Review URL: https://codereview.chromium.org/357603006

4 weeks agoFix Ninja/CMake build, broken by earlier change
Mark Seaborn [Fri, 27 Jun 2014 21:31:15 +0000 (14:31 -0700)]
Fix Ninja/CMake build, broken by earlier change

The earlier change, ed7a64c4217699967be9991e3a92df2826399ef7, added a
dependency from lib/Bitcode/NaCl/Analysis/ on the PNaCl ABI checker,
which wasn't declared.  This caused lib/libLLVMNaClBitAnalysis.so to
fail to link.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3814
TEST=build using Ninja
R=dschuff@chromium.org, jvoung@chromium.org

Review URL: https://codereview.chromium.org/341313003

4 weeks agoAllow one module block in pnacl bitcode files for pnacl-bcdis.
Karl Schimpf [Thu, 26 Jun 2014 16:10:14 +0000 (09:10 -0700)]
Allow one module block in pnacl bitcode files for pnacl-bcdis.

Also adds operator<< for class NaClBitcodeRecord.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/346763009

4 weeks agoFinish removing the finer grained Sfi* control flags
Jan Voung [Wed, 25 Jun 2014 19:28:09 +0000 (12:28 -0700)]
Finish removing the finer grained Sfi* control flags

Commit d021e666574ea564460a84588edc272eb65a4cd6 changed
checks to key off of NaCl instead, to be consistent with
X86. So nothing actually uses the flags anymore.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3858
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/346263012

4 weeks agoRemove llvm -pnaclabi-allow-dev-intrinsics flag.
Jan Voung [Wed, 25 Jun 2014 19:16:30 +0000 (12:16 -0700)]
Remove llvm -pnaclabi-allow-dev-intrinsics flag.

Last users appear to have been removed and it's removed from the
driver plumbing in: https://codereview.chromium.org/333033002/

BUG=none
R=mseaborn@chromium.org

Review URL: https://codereview.chromium.org/356563003

5 weeks agoFix issues with verifying PNaCl ABI conditions in pnacl-bcdis.
Karl Schimpf [Tue, 24 Jun 2014 18:26:43 +0000 (11:26 -0700)]
Fix issues with verifying PNaCl ABI conditions in pnacl-bcdis.

Does several things:

(1) Factors out common support functions from PNaClABIVerfiyFunctions
and PNaClABIVerifyModules, so that they can be used without a pass
manager.

(2) Fixes DataLayout used by pnacl-bcdis to match NaClBitcodeReader.

(3) Uses the functions factored out of (1) in pnacl-bcdis.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/349683004

5 weeks agoRemove special nacltrap handling from X86 NaCl.
Jan Voung [Mon, 23 Jun 2014 17:26:53 +0000 (10:26 -0700)]
Remove special nacltrap handling from X86 NaCl.

I believe it was added previously to work around the validator not
accepting ud2, so it would emit a null pointer deref instead.

However the validator is fixed now:
https://code.google.com/p/nativeclient/issues/detail?id=2094

There was some asm parsing support for parsing a "nacltrap" pseudo-inst,
but the -filetype=asm path never emitted a "nacltrap" since the nacl-as
didn't support that pseudo-inst. I.e., it was supported on the parsing
side but not the writing side. It doesn't look like anything used this
parsing support, and it wouldn't have worked with nacl-as anyway.

BUG=none
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/343273003

5 weeks agoPNaCl dynamic loading: Allow the "__pnacl_pso_root" symbol as an entry point
Mark Seaborn [Fri, 20 Jun 2014 23:11:23 +0000 (16:11 -0700)]
PNaCl dynamic loading: Allow the "__pnacl_pso_root" symbol as an entry point

Whereas pexes will allow a single external function called "_start",
PSOs (PNaCl dynamically loadable objects) will allow a single external
global variable called "__pnacl_pso_root".

Change PNaClABISimplify.cpp to allow "__pnacl_pso_root" to be left as
external.

Change the ABI verifier to allow "__pnacl_pso_root" to be defined (but
not at the same time as "_start").

Extend some tests to cover GlobalVariables as well as functions.  Make
one CHECK-NOT test stricter.

Note that this will effectively be inaccessible by PNaCl apps, until we
add an IRT interface and browser-side plumbing for loading PSOs.  If an
app were to try to load a PSO as an executable pexe, the ABI verifier
would allow the PSO, but the native-link of the PSO would fail because
the PSO wouldn't define "_start" (it would only define
"__pnacl_pso_root").

This means the change doesn't really expose any functionality to web
apps, so I don't think it's necessary to put this behind a flag.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3873
TEST=lit tests
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/331313005

5 weeks agoAdd assembly text to function blocks in pnacl-bcdis.
Karl Schimpf [Thu, 19 Jun 2014 20:33:05 +0000 (13:33 -0700)]
Add assembly text to function blocks in pnacl-bcdis.

Adds assembly text to function blocks in pnacl-bcdis.

Also constraints vector types appearing in the types block, and
defines assembly for block in the valuesymtab block.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/324433005

5 weeks agoPNaCl: Fix uninitialised StreamingMode field in PNaClABIVerifyModule
Mark Seaborn [Wed, 18 Jun 2014 15:25:39 +0000 (08:25 -0700)]
PNaCl: Fix uninitialised StreamingMode field in PNaClABIVerifyModule

This would be uninitialised when using "-verify-pnaclabi-module" with
"opt", but not when using "pnacl-abicheck".

BUG=none
TEST=ran "opt -verify-pnaclabi-module" under valgrind
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/342573003

6 weeks agoPNaCl: Make PNaClABIVerifyModule complain if _start() isn't defined
Mark Seaborn [Tue, 17 Jun 2014 22:40:21 +0000 (15:40 -0700)]
PNaCl: Make PNaClABIVerifyModule complain if _start() isn't defined

Make the ABI checker stricter so that it complains if the module doesn't
contain an entry point.

If we mistakenly internalised _start() (but left its name in place),
previously we might not have caught that because pnacl-llc
re-externalises symbols when doing streaming translation.  This new
check would catch the mistake.

Update some tests to define _start() so that they still pass.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3873
TEST=lit tests
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/330623003

6 weeks agoPNaCl translator: combine insertelement / extractelement patterns generated by the...
Derek Schuff [Tue, 17 Jun 2014 20:25:01 +0000 (13:25 -0700)]
PNaCl translator: combine insertelement / extractelement patterns generated by the toolchain into shufflevector
shufflevector isn't part of the PNaCl ABI, insertelement /
extractelement sequences are instead generated, but they help in
generating higher performance code. This pass recognizes the patterns at
translation time and reconstructs shufflevector, allowing the generated
code to perform better.

R=jvoung@chromium.org
TEST= ninja check
BUG= http://code.google.com/p/nativeclient/issues/detail?id=2205

This patch re-applies 1eecfea5ce

Review URL: https://codereview.chromium.org/341493003

6 weeks agoUpdate default mcpu values to not use SSSE3 and tune for pre-sandybridge CPUs
Derek Schuff [Tue, 17 Jun 2014 16:30:48 +0000 (09:30 -0700)]
Update default mcpu values to not use SSSE3 and tune for pre-sandybridge CPUs

The "x86-64" value is the base 64-bit target which does not
use SSE3+ as core2 did.
The pentium4m mcpu value is the same as pentium4 but uses
x86.td's  FeatureSlowBTMem attribute meaning
"Bit testing of memory is slow" which is true for pre-sandybridge
CPUs (which are probably still prevalent, and in any case
matches the 64-bit code)
BUG=https://code.google.com/p/nativeclient/issues/detail?id=2205
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/333353003

6 weeks agoMake PNaCl type checking visible so that pnacl-bcdis can use.
Karl Schimpf [Mon, 16 Jun 2014 22:04:08 +0000 (15:04 -0700)]
Make PNaCl type checking visible so that pnacl-bcdis can use.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/332813002

6 weeks agoPNaCl sandboxed translator: fix crash
JF Bastien [Fri, 13 Jun 2014 20:46:19 +0000 (13:46 -0700)]
PNaCl sandboxed translator: fix crash

I added a data layout usage to the translator [*] and broke the sandboxed
translator because the function ABI verification pass was added before the pass
manager contained a data layout, causing a default data layout to be constructed
(which errors out). This CL fixes the breakage by making it possible to use data
layout in the translator, even when sandboxed.

  [*]: https://codereview.chromium.org/321733002/

This CL also stubs out LLVM's current_path function because getcwd returns an
error when run inside the sandboxed translator (it does go through a NaCl
syscall, but we don't override it to anything meaningful for the
translator). This happens only in debug builds (which I was testing with) because
it's only asserted on (so release builds don't see this error).

R=jvoung@chromium.org, dschuff@chromium.org
TEST= ./pnacl/build.sh translator-all && ./scons bitcode=1 use_sandboxed_translator=1 small_tests
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3870

Review URL: https://codereview.chromium.org/337793002

6 weeks agoFactor out NaCl bitcode value decoders from bitcode reader.
Karl Schimpf [Thu, 12 Jun 2014 17:41:06 +0000 (10:41 -0700)]
Factor out NaCl bitcode value decoders from bitcode reader.

Factor out NaCl bitcode value decoders into a separate file, so
that both the bitcode reader and pnacl-bcdis can use them.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/327273006

6 weeks agoPNaCl SIMD: allow element-aligned vector load/store
JF Bastien [Wed, 11 Jun 2014 16:23:54 +0000 (09:23 -0700)]
PNaCl SIMD: allow element-aligned vector load/store

PNaCl currently breaks up vector load/store instructions into the corresponding
sub-elements of the vector using {insert/extract}element followed by scalar
load/store. This was originally done so that version 0 of PNaCl SIMD wouldn't
have to bother with vector load/store (especially their alignment), punting all
complexity to existing scalar instructions. This is very suboptimal
performance-wise, and re-creating the vector load/store on the translator side
isn't a trivial matter and has several caveats.

Add support for vector load/store, aligned to their element size, in PNaCl's
ABI.

R=jvoung@chromium.org
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check)
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3870

Review URL: https://codereview.chromium.org/321733002

7 weeks agoPNaCl: run constantmerge after globalizing constant vectors
JF Bastien [Wed, 11 Jun 2014 00:36:58 +0000 (17:36 -0700)]
PNaCl: run constantmerge after globalizing constant vectors

This should help reduce the number of global constants, thereby reducing the
pexe size.

R=jvoung@chromium.org
BUG= none

Review URL: https://codereview.chromium.org/325913003

7 weeks agoFix forward reference bug in pnacl-bcdis.
Karl Schimpf [Tue, 10 Jun 2014 21:09:27 +0000 (14:09 -0700)]
Fix forward reference bug in pnacl-bcdis.

Fixes bug where relocation records in the globals block can
refer to global addresses not yet defined. Previously,
the Bitcode Id for such references were incorrectly labeled
as function-level identifiers. Now, they correctly refer
to the forward referenced global address ID.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/328833002

7 weeks agoAdd PassManager to pnacl-abicheck and pnacl-llc
JF Bastien [Mon, 9 Jun 2014 23:28:09 +0000 (16:28 -0700)]
Add PassManager to pnacl-abicheck and pnacl-llc

I need to get a DataLayout in the translator (to deal with vector alignment),
and this requires a PassManager. There are other unexpected dependencies that
crop up when trying to go halfway without adding a PassManager (e.g. the
AsmPrinter is null and crashes on the mangler), but in the end the following
line is the one that requires the PassManager:
  include/llvm/PassAnalysisSupport.h:200

R=jvoung@chromium.org, dschuff@chromium.org
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check)
BUG= https://code.google.com/p/nativeclient/issues/detail?id=3870
NOTRY=true

Review URL: https://codereview.chromium.org/326513002

7 weeks agoFix block bug in "pnacl-bccompress --remove-abbreviations".
Karl Schimpf [Fri, 6 Jun 2014 21:24:07 +0000 (14:24 -0700)]
Fix block bug in "pnacl-bccompress --remove-abbreviations".

If we are removing abbreviations from a PNaCl bitcode file
(in pnacl-bccompress), be sure to reduce block abbreviation size
to only allow default abbreviations.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/320083002

7 weeks agoRetry reverting defs list for naclcalls x86-64
Jan Voung [Wed, 4 Jun 2014 23:03:58 +0000 (16:03 -0700)]
Retry reverting defs list for naclcalls x86-64

This retries the x86-64 version of commit:
bd61bacb04eaac6fdc3107136413b6ccdb9273e5.

The original change tickled a stack corruption/memory bug
exercised by an SDK test. After the test code runs and
returns, the test harness needs to get the gtest "this"
pointer from rbx. Usually rbx is saved/restored on/from
the stack.

With calls like (a) early_gtest_func -> (b) gtest_framework_func
-> (c) actual_test_func -> (d)..., the pre-patched code
used rbx in both (b) and (c), so stored a copy of rbx at
both (b) and (c). The stack corruption corrupted (c).
Since the pre-patched code still had a copy in (b), it
could use rbx fine in (a). However, the newer code
would use the corrupted version from (c) in (a).

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3867
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/310353002

8 weeks agoPNaCl: Remove warnings about TLS/init_array variables being unreferenced
Mark Seaborn [Mon, 2 Jun 2014 17:13:51 +0000 (10:13 -0700)]
PNaCl: Remove warnings about TLS/init_array variables being unreferenced

For example:

Warning: Variable __fini_array_start not referenced
Warning: Variable __fini_array_end not referenced
Warning: Variable __tls_template_start not referenced
Warning: Variable __tls_template_tdata_end not referenced
Warning: Variable __tls_template_end not referenced
Warning: Variable __tls_template_alignment not referenced

These warnings often occur when building libc-free programs.  This is
actually fairly common, so it makes sense to drop these warnings.
They're too noisy -- they hardly ever indicate a problem.

BUG=none
TEST=none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/303273005

8 weeks agoAdd assembly for abbreviations in pnacl-bcdis.
Karl Schimpf [Mon, 2 Jun 2014 16:15:34 +0000 (09:15 -0700)]
Add assembly for abbreviations in pnacl-bcdis.

Adds assembly syntax for abbreviations, and the corresponding abbreviation
index when the abbreviation is used.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/296193006

8 weeks agoRevert tablegen localmods for 32-bit naclcalls. Use normal calls.
Jan Voung [Fri, 30 May 2014 16:31:06 +0000 (09:31 -0700)]
Revert tablegen localmods for 32-bit naclcalls. Use normal calls.

There isn't much reason for these extra TD nodes.
Unlike 64-bit nacl, there isn't a difference in
pointer size in the data layout. Moreover, there are
separate non-codegen nodes for the assembler's convenience
"isAsmParserOnly = 1" nodes. Those are for parsing
"naclcall" (which aligns to end of bundle) vs "call" (which
does not align to end) in .S files. We might move to a
model where NaCl always aligns any "call" instruction it
sees anyway.

The 64-bit naclcalls are a bit hairier since we have a
different pointer size from normal X86, so can't reuse the
normal 64-bit tablegen nodes. Either pattern matching will
fail, or register copies will fail w/ things like:

mov %edi, %rax // ???
jmp *%rax

This has a side-effect of redoing the "Remove the Defs list
in X86InstrNaCl naclcall, since calls now use regmask."
CL's effects for 32-bit, to regain some of the speedups.
This also makes the x86-32 backend more similar to the ARM
backend, in that NaCl did not need a special call node.

E.g., the dip from landing and the undip from reverting:
https://chromeperf.appspot.com/report?masters=NativeClient&bots=lucid_64-newlib-x86_32-pnacl-spec&tests=naclperf%2Fcompiletime_gcc&checked=core&start_rev=13220&end_rev=13252

How to get that for 64-bit calls still TBD.

BUG=none
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/306913002

2 months agoRevert "Remove the Defs list in X86InstrNaCl naclcall, since calls now use regmask."
Jan Voung [Wed, 28 May 2014 02:12:25 +0000 (19:12 -0700)]
Revert "Remove the Defs list in X86InstrNaCl naclcall, since calls now use regmask."

This reverts commit bd61bacb04eaac6fdc3107136413b6ccdb9273e5.

It appears to be causing a miscompile in the SDK nacl_io_socket_test debug
build, leading to a segfault in some gtest code. Revert for now since
correct code is better than 5-10% faster compile time in O2.

BUG= https://code.google.com/p/chromium/issues/detail?id=377084
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/294393006

2 months agoPNaCl: Fix setjmp() by adding the "returns_twice" attribute back
Mark Seaborn [Sun, 25 May 2014 00:21:02 +0000 (17:21 -0700)]
PNaCl: Fix setjmp() by adding the "returns_twice" attribute back

This fixes a bug in which stack slots could get reused by the backend.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3733
TEST=lit test + run_longjmp_stackslots_test being added to NaCl Scons tests
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/293313002

2 months agoNaCl: Fix memset() for Non-SFI ARM
Mark Seaborn [Fri, 23 May 2014 20:51:13 +0000 (13:51 -0700)]
NaCl: Fix memset() for Non-SFI ARM

The backend was calling memset() but using the calling conventions for
__aeabi_memset(), which swap two of the arguments.  We need to extend
the existing NaCl localmod to fix this.

Declare EnableARMAEABIFunctions so that another file can refer to it.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3809
TEST=run_memcpy_move_set_test_irt + new lit test
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/291333002

2 months agoFix compiler error for mac-pnacl-x86_32 for CL 298803005.
Karl Schimpf [Fri, 23 May 2014 20:10:00 +0000 (13:10 -0700)]
Fix compiler error for mac-pnacl-x86_32 for CL 298803005.

src/llvm/lib/Bitcode/NaCl/Analysis/NaClObjDump.cpp:1640:
error: no matching function for call to
'<unnamed>::SignRotatedInt::SignRotatedInt(<unnamed>::SignRotatedInt)'

BUG=None
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/292363007

2 months agoAdd assembly code for the constants block in pnacl-bcdis.
Karl Schimpf [Fri, 23 May 2014 17:49:04 +0000 (10:49 -0700)]
Add assembly code for the constants block in pnacl-bcdis.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/298803005

2 months agoRemove writing useless abbreviation by the PNaCl bitcode writer.
Karl Schimpf [Fri, 23 May 2014 17:33:35 +0000 (10:33 -0700)]
Remove writing useless abbreviation by the PNaCl bitcode writer.

That is: TYPE_POINTER_ABBREV  [8, fixed(3), 0]

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/294213003

2 months agoFix include file dependency in NaClObjDumpStream.h
Karl Schimpf [Fri, 23 May 2014 16:54:58 +0000 (09:54 -0700)]
Fix include file dependency in NaClObjDumpStream.h

BUG= None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/299923002

2 months agoAdd support for building subzero with LLVM
Derek Schuff [Thu, 22 May 2014 23:22:13 +0000 (16:22 -0700)]
Add support for building subzero with LLVM

Since subzero depends on LLVM components, it makes sense to build it
along with the LLVM build. Also, toolchain_build doesn't have support
for a target to depend on another target's working directory. We may want
to fix that in the future, but for now it's simpler to do the build
together with LLVM.

This CL allows subzero to be built from an external directory similar to
the way clang is. However we just use a Make variable directly insetad
of a configure flag to avoid regenerating autoconf files.

R=jvoung@chromium.org, stichnot@chromium.org
BUG=

Review URL: https://codereview.chromium.org/299753004

2 months agoAbstract out memory pool notion from PNaCl text formatter.
Karl Schimpf [Thu, 22 May 2014 22:34:31 +0000 (15:34 -0700)]
Abstract out memory pool notion from PNaCl text formatter.

The PNaCl bitcode objdumper defines several simple memory pools for
different kinds of token directives. Abstracts out this concept
with a template, simplifying the code.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/298683007

2 months agoRemoving FlagSfi* in LLVM -- check Subtarget->isTargetNaCl() instead.
Jan Voung [Thu, 22 May 2014 22:27:28 +0000 (15:27 -0700)]
Removing FlagSfi* in LLVM -- check Subtarget->isTargetNaCl() instead.

We can't remove the flags just yet because the driver still refers to them.
So, as a first step just don't use the flag. Then after a deps roll, I'll remove
the references from the driver, and then we can remove the flags themselves.

For the constant pool disabling, check useConstIslands
instead of FlagSfiDisableCP, and initialize UseConstIslands
appropriately for NaCl.

Had to adjust some of the tests slightly:
(*) some did not expect sandboxing, but expected a specific instruction seq
(*) one fast-isel case only handled static relocations w/ constant islands disabled.

Remove FlagSfiZeroMask right now, since there are no users
in NaCl repo. The other flags are still there because the
NaCl repo's pnacl-translate.py still refers to them.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3858
R=dschuff@chromium.org, jfb@chromium.org

Review URL: https://codereview.chromium.org/293063007

2 months agoAdd recognition of abbreviation records in pnacl-bcdis.
Karl Schimpf [Wed, 21 May 2014 21:36:55 +0000 (14:36 -0700)]
Add recognition of abbreviation records in pnacl-bcdis.

Adds code to uses a listener to capture abbreviations in
pnacl-bcdis.

Also discovered that ExitBlock's were not being handled
very consistenly (between using a Listener or not) when
using class NaClBitcodeParser. Cleaned up this issue.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/289033003

2 months agoRemove the Defs list in X86InstrNaCl naclcall, since calls now use regmask.
Jan Voung [Wed, 21 May 2014 15:27:47 +0000 (08:27 -0700)]
Remove the Defs list in X86InstrNaCl naclcall, since calls now use regmask.

LLVM r150638 made it so that call instructions no longer need the list
of all call-clobbered registers:
https://llvm.org/viewvc/llvm-project?view=revision&revision=150638

Make the NaCl variants of calls more like the LLVM
variants. At some point perhaps the special calls
can be removed an just use the normal calls w/ the
appropriate operand types.

This should make things a bit faster w/ NaCl (get the gains of switching
to the regmask vs the long list of defs). E.g.,:

0.38  seconds to  compile gzip vs 0.4 seconds.

BUG=none (misc cleanup)
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/288113006

2 months agoExtract out PNaCl objdumper into a library file.
Karl Schimpf [Tue, 20 May 2014 20:51:57 +0000 (13:51 -0700)]
Extract out PNaCl objdumper into a library file.

This is in preparation of adding error examples for the PNaCl
objdumper. By being in a library, we can add unit tests that
mangle a sequence of records in the bitcode file, and see
what happens when dumped.

BUG= https://codereview.chromium.org/279523004
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/290843004

2 months agoGeneralize pnacl-bccompress to be able to remove existing abbreviations.
Karl Schimpf [Tue, 20 May 2014 20:45:35 +0000 (13:45 -0700)]
Generalize pnacl-bccompress to be able to remove existing abbreviations.

Adds a flag to pnacl-bccompress. When true, the input file is copied but
all abbreviations are removed. This allows us to generate simple
bitcode files (for examples in the PNaCl bitcode file ABI), as well
as determine how effective existing abbreviations are in pnacl-freeze.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/290833002

2 months agoRevert "PNaCl translator: combine insertelement / extractelement patterns generated...
Derek Schuff [Tue, 20 May 2014 16:47:47 +0000 (09:47 -0700)]
Revert "PNaCl translator: combine insertelement / extractelement patterns generated by the toolchain into shufflevector"

This reverts commit 1eecfea5ce79d611558e8c132944fe752cca8278.

Many (most?) try jobs are failing the shufflevector test now, whereas they were
mostly (all?) passing before.

Original review:
  https://codereview.chromium.org/282553003/

R=jfb@chromium.org
BUG= https://code.google.com/p/nativeclient/issues/detail?id=2205

Review URL: https://codereview.chromium.org/294943004

2 months agoAdd abbreviation index to records in pnacl-bcdis output.
Karl Schimpf [Mon, 19 May 2014 16:47:17 +0000 (09:47 -0700)]
Add abbreviation index to records in pnacl-bcdis output.

The objdumper did not show the abbreviation index used. Add
this information into the output. To make sure that abbreviation
indices are not confused with record address, the record addresses
are put in a separate column.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/291513002

2 months agoImprove readability of valuesymtab blocks in PNaCl bitcode.
Karl Schimpf [Mon, 19 May 2014 16:13:57 +0000 (09:13 -0700)]
Improve readability of valuesymtab blocks in PNaCl bitcode.

If the name of a symbol is not alphanumeric (i.e. Char6) in a
valuesymtab block of PNaClBitcode. Print all Char6 characters
as characters, so that it is easier to read.

BUG=None
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/290823002

2 months agoAdd assembly code for the globals block in pnacl-bcdis.
Karl Schimpf [Mon, 19 May 2014 16:08:17 +0000 (09:08 -0700)]
Add assembly code for the globals block in pnacl-bcdis.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/264973004

2 months agoGenerate assembly instructions for value symtab block in pnacl-bcdis.
Karl Schimpf [Fri, 16 May 2014 17:41:11 +0000 (10:41 -0700)]
Generate assembly instructions for value symtab block in pnacl-bcdis.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/279523004

2 months agoRemove nosys from llvm sandboxed build.
Jan Voung [Fri, 16 May 2014 17:07:22 +0000 (10:07 -0700)]
Remove nosys from llvm sandboxed build.

It's just an empty library now. It might be the last
reference to libnosys in scons. (sdk still references gyp)

Also remove -lnacl and -lm since those will automatically get
pulled in by pnacl-clang++ when linking.

BUG=none
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/267443004

2 months agoPNaCl translator: combine insertelement / extractelement patterns generated by the...
JF Bastien [Mon, 12 May 2014 22:48:17 +0000 (15:48 -0700)]
PNaCl translator: combine insertelement / extractelement patterns generated by the toolchain into shufflevector

shufflevector isn't part of the PNaCl ABI, insertelement /
extractelement sequences are instead generated, but they help in
generating higher performance code. This pass recognizes the patterns at
translation time and reconstructs shufflevector, allowing the generated
code to perform better.

R=dschuff@chromium.org
TEST= ninja check
BUG= http://code.google.com/p/nativeclient/issues/detail?id=2205

Review URL: https://codereview.chromium.org/282553003

2 months agoNaClAbbrevListener: add missing virtual dtor
JF Bastien [Wed, 7 May 2014 22:29:00 +0000 (15:29 -0700)]
NaClAbbrevListener: add missing virtual dtor

It was generating a warning at build time.

R= kschimpf@chromium.org
BUG= warning at build time

Review URL: https://codereview.chromium.org/274563004

2 months agoPNaCl SIMD: allow the bitcode reader to read select instructions properly
JF Bastien [Wed, 7 May 2014 18:33:52 +0000 (11:33 -0700)]
PNaCl SIMD: allow the bitcode reader to read select instructions properly

Vector selects weren't handled properly by the NaCl bitcode reader but
were handled properly by the LLVM one: the addition of vectors was
missing tests for select with a vector mask, which had (correctly) been
removed from the NaCl bitcode reader but not added back when vector
support was added.

R=dschuff@chromium.org, kschimpf@chromium.org
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check-all)
BUG= https://code.google.com/p/nativeclient/issues/detail?id=2205

Review URL: https://codereview.chromium.org/270723003

2 months agoAdd check that expected number of types match found types in pnacl-bcdis
Karl Schimpf [Tue, 6 May 2014 20:46:03 +0000 (13:46 -0700)]
Add check that expected number of types match found types in pnacl-bcdis

BUG=None
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/265793008

2 months agoAdd virtual destructor to class NaClBitcodeParserListener.
Karl Schimpf [Tue, 6 May 2014 20:43:57 +0000 (13:43 -0700)]
Add virtual destructor to class NaClBitcodeParserListener.

Removes compliation warning message about the class using
virtual methods, but not defining a virtual destructor.

BUG=None
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/262243003

2 months agoLLVM biased sandbox build: fix build issues
JF Bastien [Tue, 6 May 2014 17:31:44 +0000 (10:31 -0700)]
LLVM biased sandbox build: fix build issues

All minor issues: missing else, signed/unsigned mismatch warning, unused
variable, and lack of __clear_cache function (which shouldn't be called
from NaCl anyways).

The goal of a biased sandbox build is to allow the sandboxed translator
to call CPUID on x86 so that it can figure which SIMD instructions can
be generated.

R=dschuff@chromium.org, jvoung@chromium.org
TEST= ./pnacl/build.sh translator-all
BUG= https://code.google.com/p/nativeclient/issues/detail?id=2205

Review URL: https://codereview.chromium.org/264893005

2 months agoPNaCl: Add option to turn off use of __aeabi_* functions for Non-SFI NaCl
Mark Seaborn [Mon, 5 May 2014 23:18:09 +0000 (16:18 -0700)]
PNaCl: Add option to turn off use of __aeabi_* functions for Non-SFI NaCl

Currently, PNaCl's arm-nonsfi target uses the triple
"armv7a-linux-none", which uses soft-float, which isn't the right ABI
for Non-SFI NaCl.

Switching to a triple that uses hard-float
(e.g. "armv7a-linux-gnueabihf") has the side effect of causing llc to
generate uses of __aeabi_* functions, which PNaCl's libgcc.a (from
compiler-rt) doesn't provide.

In the longer term, the cleaner way to address that is to build
compiler-rt to provide __aeabi_* functions, but doing that for SFI NaCl
would involve porting some ARM assembly code to SFI NaCl, and that might
be a minor performance regression -- e.g. lib/arm/aeabi_uidivmod.S
defines __aeabi_uidivmod() as a wrapper for __udivmodsi4().

For now, just extend the existing localmod so that we can use it for
Non-SFI NaCl too.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3809
TEST=also tested with pnacl-translate.py change
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/266913009

2 months agoFix bug introduced by method rename for pnacl-bcdis.
Karl Schimpf [Thu, 1 May 2014 21:39:12 +0000 (14:39 -0700)]
Fix bug introduced by method rename for pnacl-bcdis.

Fix bug introduced into pnacl-bcdis, due to separate CL landing near the
same time, and that CL renamed method to ProcessBlockInfo.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/260903012

2 months agoInitial implementation of pnacl-bcdis.
Karl Schimpf [Thu, 1 May 2014 15:52:14 +0000 (08:52 -0700)]
Initial implementation of pnacl-bcdis.

Defines a text form for records in PNaCl bitcode. Currently excludes
adding abbreviation information. Text of instructions etc. are based
on LLVM usage.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/201723007

2 months agoAdd listener to PNaCl bitstream reader to capture abbreviations.
Karl Schimpf [Wed, 30 Apr 2014 21:38:58 +0000 (14:38 -0700)]
Add listener to PNaCl bitstream reader to capture abbreviations.

Defines a listener to capture abbreviations read by the PNaCl
bitstream reader, so that PNaCl bitcode parsers can use this
information during analysis.

Also simplifies pnacl-bccompress by using this API to capture
abbreviations as they are read.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/254713009

2 months agoPNaCl SIMD: expand shufflevector
JF Bastien [Wed, 30 Apr 2014 01:27:44 +0000 (18:27 -0700)]
PNaCl SIMD: expand shufflevector

This allows hand writing code that used __builtin_shufflevector or
generating LLVM bitcode that uses shufflevector.

Corresponding end-to-end test:
  https://codereview.chromium.org/255213002

R=dschuff@chromium.org, nfullagar@chromium.org
BUG= https://code.google.com/p/nativeclient/issues/detail?id=2205
TEST=  (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check) && ./scons run_vector_shuffle_test bitcode=1

Review URL: https://codereview.chromium.org/256233003

3 months agoAllow nested clustering in the NaCl text formatter.
Karl Schimpf [Mon, 28 Apr 2014 15:41:09 +0000 (08:41 -0700)]
Allow nested clustering in the NaCl text formatter.

Generalizes the notion of one level clustering to arbitrarily nested
clustering in the NaCl text formatter. Also fixes bad interaction
between local indentations and clustering in the old implementation.

Includes unit tests for NaCl text formatter.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/247073011

3 months agoPNaCl vectors: disallow zeroinitializer, globalize them instead
JF Bastien [Tue, 22 Apr 2014 20:25:36 +0000 (13:25 -0700)]
PNaCl vectors: disallow zeroinitializer, globalize them instead

The bitcode reader/write didn't allow them, and small tests hadn't run into this issue yet.

R=jvoung@chromium.org, dschuff@chromium.org
TEST= make check
BUG= https://code.google.com/p/nativeclient/issues/detail?id=2205

Review URL: https://codereview.chromium.org/247533003

3 months agopnacl-abicheck: initialize function pass before running it
JF Bastien [Mon, 21 Apr 2014 18:58:19 +0000 (11:58 -0700)]
pnacl-abicheck: initialize function pass before running it

PNaClABIVerifyFunctionsPass wasn being initialized, causing it to crash.

R=dschuff@chromium.org
TEST= pnacl-abicheck *.pexe
BUG= none

Review URL: https://codereview.chromium.org/245613002

3 months agoRevert localmod for re-ordering -lLTO on link line for gold plugin.
Jan Voung [Mon, 21 Apr 2014 18:28:23 +0000 (11:28 -0700)]
Revert localmod for re-ordering -lLTO on link line for gold plugin.

We had a localmod fix for moving -lLTO to the end of the link line
but that was also done upstream, so the localmod is redundant:
http://llvm.org/bugs/show_bug.cgi?id=13262

vs

https://codereview.chromium.org/10693178/

BUG=none
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/245343002

3 months agoRevert localmod workaround for dangling debug info assertion.
Jan Voung [Mon, 21 Apr 2014 16:22:09 +0000 (09:22 -0700)]
Revert localmod workaround for dangling debug info assertion.

Underlying problem may have been fixed with LLVM 3.4
http://llvm.org/bugs/show_bug.cgi?id=11225.

Was able to link, optimize, and translate pnacl-llc.pexe
and gold.pexe w/ -g, without asserting on shutdown as
before. Also tried dosbox and nacl SDK earth example,
but not the ARM IRT (previous test case) since that isn't
built by PNaCl anymore.

TEST=none (could never get a small repro because bugpoint
isn't useful for minimizing things with debug metadata).

BUG= http://code.google.com/p/nativeclient/issues/detail?id=2786
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/243883003

3 months agoPNaCl: Add support for GCC/LLVM vector extensions
JF Bastien [Wed, 16 Apr 2014 23:18:02 +0000 (16:18 -0700)]
PNaCl: Add support for GCC/LLVM vector extensions

Add limited SIMD support for 128-bit wide SIMD for integers of size
8/16/32 and 32-bit floating-point, and operations that can be expressed
through GCC/LLVM vector extensions:
 *  http://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html
 *  http://clang.llvm.org/docs/LanguageExtensions.html#vectors-and-extended-vectors

64-bit integers and floating-point are currently missing and will be
added in a later CL.

Boolean vectors are represented as one bit per vector element, and the
number of elements in a boolean vector matches that of the supported
types (4, 8, 16).

256-bit and 512-bit vectors will take some work to support and may not
be supported in the near future.

Supported operations are:
  unary +, -
  ++, --
  +, -, *, /, %
  &, |, ^, ~
  >>, <<
  !, &&, ||
  ==, !=, >, <, >=, <=
  =

Vector types are usually defined in C/C++ with:
  __attribute__((vector_size(VEC_BYTES)))

Note that this includes some conversions, but only to/from supported
types.

Note that this CL doesn't specify all undefined behavior, and future CLs
will ensure that undefined behavior for vectors matches that of
scalars. Worth noting are div/mod by zero (specified in PNaCl as
trapping), shift by bitwidth or greater (currently unspecified, see bug
3604), out-of-bounds conversions, signed over/underflow.

The CL does ensure that undefined behavior due to alignment is the same
as for scalars by breaking up vector load/store into element-wise scalar
load/store. This pattern should be recognized by backends which support
unaligned vector load/store, but no effort was put into making sure that
this is the case.

Undefined behavior due to out-of-bounds vector indexing is eliminated by
going through the stack. This should also be recognized by backends.

Vector constant immediate aren't supported, and instead go through loads
from the constant global array.

No intrinsics on vectors are currently supported, they will be added by
later CLs.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=2205

Testing:
TEST= (cd ./toolchain_build/out/llvm_i686_linux_work/ && ninja check)
TEST= ./scons run_simd_ref_test bitcode=1 && ./scons run_simd_native_test bitcode=1
 * I also have an extra scons test for which I is in a separate CL:
   - https://codereview.chromium.org/222483002
 * We should also make sure that the GCC torture tests for vector extensions are enabled.

Reviewers:
 * For PNaCl bitcode reader/writer:
    - kschimpf
 * For PNaCl ABI and general passes:
    - jvoung, mseaborn, dschuff
 * For testing and vector support:
    - nfullagar
 * FYI on ABI change for Subzero:
    - stichnot, eliben, sehr

R=dschuff@chromium.org, jvoung@chromium.org, eliben@chromium.org, kschimpf@chromium.org, mseaborn@chromium.org, nfullagar@chromium.org, sehr@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/221693002

3 months agoClean up handling of the previous token in NaCl Objdump.
Karl Schimpf [Wed, 16 Apr 2014 19:41:12 +0000 (12:41 -0700)]
Clean up handling of the previous token in NaCl Objdump.

Clean up the handling of the previous token in NaCl Objdump, by
having the top-level (directive) Apply method handle the text
in the token buffer, followed by a call to the directive specific
method MyApply.

Also removes access to several (otherwise protected) formatter methods,
since they are internals the directives should not be using.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/236203023

3 months agoAdd a text formatter to NaClObjDumpStream to handle long lines.
Karl Schimpf [Tue, 15 Apr 2014 21:29:42 +0000 (14:29 -0700)]
Add a text formatter to NaClObjDumpStream to handle long lines.

Provide a simple facility to NaClObjDumpStream to handle long
lines. This is done by only requiring the user of an text formatter to
tokenize the printed text, and let the text formatter class
automatically handle insertion of newlines (and beginning of line
indentation).

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/225263008

3 months agoRemove LLVM's StreamInit/StreamInitWithOverrides. No longer used.
Jan Voung [Mon, 14 Apr 2014 21:53:23 +0000 (14:53 -0700)]
Remove LLVM's StreamInit/StreamInitWithOverrides. No longer used.

The drivers and chrome have changed to only use the
version that can support split modules (1 up to 16 threads).
E.g., from https://codereview.chromium.org/197023007

BUG=none
R=dschuff@chromium.org

Review URL: https://codereview.chromium.org/237393004

3 months agoPNaCl bitcode reader: restrict some record sizes
JF Bastien [Wed, 2 Apr 2014 23:25:43 +0000 (16:25 -0700)]
PNaCl bitcode reader: restrict some record sizes

The PNaCl bitcode reader allows too many entries in some records, and
these extra entries never happen in practice. This change makes sure we
don't allow and then ignore too many things in the bitcode format.

This came up in:
  https://codereview.chromium.org/221693002/

R=kschimpf@google.com, jvoung@chromium.org, kschimpf@chromium.org
TEST= ./scons small_tests medium_tests bitcode=1
BUG= none

Review URL: https://codereview.chromium.org/222313004

3 months agoFlattenGlobals: Only check size and alignment for non-opaque variables.
Jan Voung [Tue, 1 Apr 2014 17:55:04 +0000 (10:55 -0700)]
FlattenGlobals: Only check size and alignment for non-opaque variables.

For opaque-typed variables leave it as zero sized.

After the 3.4 merge, some libcxx tests with dead
variables have opaque types after linking instead of
having real types. In 3.3 the dead variable did link in
the actual type.

The dead variable would be removed after the FlattenGlobals
pass, but before that it ends up asserting in DataLayout
while trying to determine the size and alignment.

BUG=http://code.google.com/p/nativeclient/issues/detail?id=3804
R=mseaborn@chromium.org

Review URL: https://codereview.chromium.org/220203002

4 months agoPNaCl: Remove all whitespace-only localmods
Mark Seaborn [Fri, 28 Mar 2014 21:29:38 +0000 (14:29 -0700)]
PNaCl: Remove all whitespace-only localmods

This removes all localmod patch hunks that only add or remove empty
lines, since these can cause merge conflicts when merging from
upstream and they don't provide any benefit.

These changes were presumably left behind from localmods that were
later removed (either explicitly or as a result of merging an
upstreamed patch).

BUG=none
TEST=git diff $(git merge-base upstream/master HEAD) | python find_whitespace_patches.py
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/217183005

4 months agoFix two trivial bugs in pnacl tools.
Karl Schimpf [Thu, 27 Mar 2014 19:47:52 +0000 (12:47 -0700)]
Fix two trivial bugs in pnacl tools.

1) Fix format string from "\%" to "%%", so that it prints "%"
   in pnacl-bccompress.cpp.

2) Add return value to Error method of naclbitc::ObjDumpStream.

BUG=None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/215043005

4 months agoAdd ability to print abbreviation usage in pnacl-bccompress.
Karl Schimpf [Thu, 27 Mar 2014 17:10:11 +0000 (10:10 -0700)]
Add ability to print abbreviation usage in pnacl-bccompress.

This is being added so that we can analyze common (compressed)
PNaCl bitcode files, to find common abbreviations that could
be moved into the bitcode writer.

BUG= None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/212983005

4 months agoCreate notion of objdump stream for PNaCl bitcode files.
Karl Schimpf [Thu, 27 Mar 2014 17:05:14 +0000 (10:05 -0700)]
Create notion of objdump stream for PNaCl bitcode files.

Creates a objdump stream for use with dumping PNaCl bitcode files.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3814
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/210973008

4 months agoPNaCl bitcode writer: improve error message.
JF Bastien [Thu, 27 Mar 2014 16:33:10 +0000 (09:33 -0700)]
PNaCl bitcode writer: improve error message.

The error message when writing an instruction the NaCl writer didn't like was 'LLVM ERROR: Illegal instruction' which ins't quite true since the instruction is legal for LLVM, just not for the reduced NaCl ABI. Make that clearer.

R=kschimpf@google.com, kschimpf@chromium.org
TEST= none
BUG= none

Review URL: https://codereview.chromium.org/214453004

4 months agoAdd PNaCl intrinsic declarations up front, before the function passes.
Jan Voung [Wed, 26 Mar 2014 22:40:14 +0000 (15:40 -0700)]
Add PNaCl intrinsic declarations up front, before the function passes.

Remove extra checks that were thrown off by adding decls during
the function passes.

BUG=http://code.google.com/p/nativeclient/issues/detail?id=3823
R=jfb@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/211413005

4 months agoRevert localmods to llvm-extract and ExtractGV.
Jan Voung [Sat, 22 Mar 2014 00:53:21 +0000 (17:53 -0700)]
Revert localmods to llvm-extract and ExtractGV.

This reverts https://chromiumcodereview.appspot.com/10581042/
which was an early experiment for splitting modules (but done
on the developer side).

BUG= none
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/208383003

4 months agoFix the PNaCl cmake ninja build
JF Bastien [Fri, 21 Mar 2014 23:03:54 +0000 (16:03 -0700)]
Fix the PNaCl cmake ninja build

The build was broken with an inverted dependency, and a missing dependency.

R=jvoung@chromium.org, kschimpf@chromium.org
TEST= ./toolchain_build/toolchain_build_pnacl.py --verbose --cmake --install ./toolchain/pnacl_linux_x86/
BUG= none

Review URL: https://codereview.chromium.org/201443006

4 months agoPNaCl ABI module verifier doesn't accept ConstantDataVector
JF Bastien [Fri, 21 Mar 2014 22:12:59 +0000 (15:12 -0700)]
PNaCl ABI module verifier doesn't accept ConstantDataVector

The check on isSimpleElement already precludes anything that isn't an
ArrayType, it seems like a simple oversight to check for
ConstantDataSequential (the supertype of ConstantDataArray and
ConstantDataVector).

R=mseaborn@chromium.org, jvoung@chromium.org
TEST= none
BUG= none

Review URL: https://codereview.chromium.org/209083002

4 months agoPNaCl/LLVM: add support for symbol wrapping through -Wl,--wrap
JF Bastien [Fri, 21 Mar 2014 16:57:53 +0000 (09:57 -0700)]
PNaCl/LLVM: add support for symbol wrapping through -Wl,--wrap

Support was removed a while ago because it was only used in the
now-deprecated nacl-mounts, otherwise completely untested, and based on
local modifications to Gold and LLVM.

Symbol wrapping is nonetheless used by some SDK users, and works with
GCC and regular LLVM, it would therefore be nice for it to work with
PNaCl.

The reverts, from which I cherry-picked the symbol wrapping code and
rebased to the latest Gold and LLVM:
  https://codereview.chromium.org/22831032
  https://codereview.chromium.org/119203005
  https://codereview.chromium.org/23398002

The latest bug in which this code was changed (before deletion):
  https://code.google.com/p/nativeclient/issues/detail?id=2359

For reference, nacl_io wraps Newlib APIs in a completely different way:
  https://chromium.googlesource.com/experimental/chromium/src/+/refs/heads/master/native_client_sdk/src/libraries/nacl_io/kernel_wrap_newlib.cc

This is a four part change:
  LLVM: https://codereview.chromium.org/203623007
  Gold: https://codereview.chromium.org/197883026
  NaCl test: https://codereview.chromium.org/196943027
  pnacl-ld: https://codereview.chromium.org/205153004

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3818
TEST= ./scons run_wrap_test bitcode=0 && ./scons run_wrap_test bitcode=1
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/203623007

4 months agoMoves abbreviation size computations to abbreviation blocks.
Karl Schimpf [Fri, 21 Mar 2014 16:03:11 +0000 (09:03 -0700)]
Moves abbreviation size computations to abbreviation blocks.

That is, from class NaClBitcodeCopyParser to BlockAbbreviations. This
cleans up and localizes these compuations to the best fitting class.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3720
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/205293002

4 months agoAdd self-scheduling to threaded translation (vs static)
Jan Voung [Thu, 20 Mar 2014 23:16:54 +0000 (16:16 -0700)]
Add self-scheduling to threaded translation (vs static)

For the x86-64 sandboxed translator it makes about a 5%
difference over a set of 13 pexes from the web / webstore.
For python.pexe this reduces the translation time w/ 4
threads from 43 seconds to 37 seconds. Most of the other
web benchmarks only get slight improvements (if any).

For the ARM sandboxed translator, it's about 4.5%
aggregate difference for on the a15. Again, most of the
apps are a wash except for python, gamecake and the
gameboy emulator.

For the unsandboxed translator, this is often a wash,
except on a few benchmarks.

Testing on spec2k: Same or better for sandboxed translator.
Notably, crafty translation time drops from 1.59 secs to
1.19 secs w/ CPU utilization going from 1.8 to 2.7
w/ 4 threads. With two threads it's 2.1 secs to 1.6 secs
and CPU utilization from 1.2 to 1.7. Crafty was one of
the benchmarks that regressed after switching from 3.3
to 3.4 and perhaps part of it was bad scheduling.
The gcc and eon benchmarks also get a nice boost.
Similar on ARM: crafty drops from 5.7 to 4.8 w/ 2 threads.

Use a ChunkSize of 8, to reduce the synchronization
overhead somewhat, hopefully without hurting streaming
translation. This tapers off to 1 when there are
few functions remaining to attempt finer grained load
balancing near the end.

For some idea of how this affects streaming translation
the average function size for the benchmarks range
from 160 bytes to 1700 bytes. So 8 times that is
about 1.3KB to 14KB. Geomean of average func size is
570 bytes. For that geomean, 4 threads times 8 functions
would grab ~18KB at a time on geo-average-ish.

BUG= http://code.google.com/p/nativeclient/issues/detail?id=3777
R=jfb@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/196793026

4 months agoRemoves untracked values distribution from the value size distribution.
Karl Schimpf [Wed, 19 Mar 2014 20:29:58 +0000 (13:29 -0700)]
Removes untracked values distribution from the value size distribution.

Removes the untracked values distribution from the value size
distribution, because it is never used. This speeds up pnacl-compress
by 5 to 8%.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3720
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/204103010

4 months agoMove bitcode analysis files to corresponding library.
Karl Schimpf [Wed, 19 Mar 2014 19:50:57 +0000 (12:50 -0700)]
Move bitcode analysis files to corresponding library.

Moves analysis files from lib/Bitcode/NaCl/Reader to
lib/Bitcode/NaCl/Analysis.

BUG= None
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/203263008

4 months agoFix the PNaCl ninja build
JF Bastien [Wed, 19 Mar 2014 00:24:34 +0000 (17:24 -0700)]
Fix the PNaCl ninja build

The abbrev tries test patch (https://codereview.chromium.org/183513002/) was missing two dependencies, which failed compilation (see comment in that codereview thread). This patch fixes the problem.

R=jvoung@chromium.org, kschimpf@chromium.org
TEST= ./toolchain_build/toolchain_build_pnacl.py --verbose --cmake --install ./toolchain/pnacl_linux_x86/ llvm_i686_linux
BUG= none

Review URL: https://codereview.chromium.org/204153002

4 months agoSpeed up abbreviation index selection in pnacl-bccompress.
Karl Schimpf [Wed, 12 Mar 2014 14:45:27 +0000 (07:45 -0700)]
Speed up abbreviation index selection in pnacl-bccompress.

The old implementation would try every abbreviation in a block to
find the best abbreviation to apply. This works well when there
are only a few dozen abbreviations. However, after applying
pnacl-bccompress, the number of abbreviations can exceed 1000 for
a block. Hence, a better solution is needed. This code builds
tries, based on constants that appear in abbreviations, to filter
down what abbreviations can be applied to a bitcode record.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3720
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/183513002

4 months agoLink LLVM bitcode reader into the sandboxed PNaCl for debug support.
Jan Voung [Tue, 11 Mar 2014 18:30:52 +0000 (11:30 -0700)]
Link LLVM bitcode reader into the sandboxed PNaCl for debug support.

Having both the PNaCl reader and the LLVM reader allows
developers to debug LLVM-flavored bitcode files with debug
metadata directly in the browser once developers opt-in.

Unify the interface of get*StreamedBitcodeModule between
NaCl and LLVM to both take a StreamingMemoryObject. For
LLVM, this must be a StreamingMemoryObject instead of a
StreamableMemoryObject, since LLVM uses the extra couple of
methods to skip through bitcode wrappers.

Disable Metadata ABI check when user opts in to LLVMFormat
(default is PNaClFormat for sandboxed translator).

Adds about 150KB (BitcodeReader, BitstreamReader, and
extra data, etc.) to each translator nexe.

BUG= https://code.google.com/p/nativeclient/issues/detail?id=3765

TEST= coming up, but:

pnacl/test.sh test-x86-32-sbtc gdb_tests
pnacl/test.sh test-x86-64-sbtc gdb_tests
pnacl/test.sh test-arm-sbtc gdb_tests

w/ the following hacky patch: https://codereview.chromium.org/193403002/

Also try manually compiling chromium/src/out/Release/ppapi_nacl_tests_newlib.pexe.debug

R=jfb@chromium.org, kschimpf@google.com

Review URL: https://codereview.chromium.org/189543018

4 months agoOne more mention of reduce-memory-footprint to remove.
Jan Voung [Tue, 4 Mar 2014 22:48:41 +0000 (14:48 -0800)]
One more mention of reduce-memory-footprint to remove.

Forgot to remove from previous CL.
Also reduce set of flags used to just -relocation-model
and -sfi-disable-cp, etc. to disable constant pools.
Clean up a bit too, now that CHECK-LABEL and llvm-objdump
are nicer.

BUG=none
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/185543019

4 months agoMake ReduceMemoryFootprint the default when streaming (w/ LazyBitcode).
Jan Voung [Tue, 4 Mar 2014 21:34:20 +0000 (13:34 -0800)]
Make ReduceMemoryFootprint the default when streaming (w/ LazyBitcode).

This overlaps mostly with LazyBitcode, and LazyBitcode
is flipped on in most cases we care about. Also reducing
memory footprint is a good thing to have by default
(already set by default for pnacl-translate.py, but
not for in-browser translator).

The driver change https://codereview.chromium.org/178043007/
must go in before this, otherwise there will still be
a reference to the flag. See driver CL description for more
notes...

BUG=none
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/177073023

4 months agoFix PNaCl-local files after merging LLVM 3.4 mseaborn/merge-34-squashed
Mark Seaborn [Sat, 1 Mar 2014 18:34:38 +0000 (10:34 -0800)]
Fix PNaCl-local files after merging LLVM 3.4

These are mostly compile fixes:

 * Update for interface changes:
    * readBytes() (unused arg was removed)
    * tool_output_file takes different flags
    * Materialize() uses error_code instead of bool + string
 * Remove uses of case ranges, which were removed in 3.4
 * pnacl-llc: Some TargetOptions fields were removed in 3.4

Also update test expectations.  LLVM's 'not' tool now treats crashes
as a failure unless '--crash' is passed.

BUG=https://code.google.com/p/nativeclient/issues/detail?id=3757
TEST=PNaCl toolchain trybots

Review URL: https://codereview.chromium.org/180483005/