Summary:
Some flags accepted by --set-section-flags and --rename-section can change a SHT_NOBITS section to a SHT_PROGBITS section. Note that none of them can change a SHT_PROGBITS to SHT_NOBITS.
The full list (found via experimentation of individually setting each flag) that does this is: contents, load, noload, code, data, rom, and debug.
This was found by testing llvm-objcopy with the gnu binutils test suite, specifically this test case: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=binutils/testsuite/binutils-all/copy-1.d;h=f2b0d9e90df738c2891b4d5c7b62f62894b556ca;hb=HEAD
Reviewers: jhenderson, grimar, jakehehrlich, alexshap, espindola
Reviewed By: jhenderson
Subscribers: emaste, arichardson, MaskRay, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D59958
llvm-svn: 357492
This patch fixes https://bugs.llvm.org/show_bug.cgi?id=41293 and
https://bugs.llvm.org/show_bug.cgi?id=41045. llvm-objcopy assumed that
it could always read a section header string table. This isn't the case
when the sections were previously all stripped, and the e_shstrndx field
was set to 0. This patch fixes this. It also fixes a double space in an
error message relating to this issue, and prevents llvm-objcopy from
adding extra space for non-existent section headers, meaning that
--strip-sections on the output of a previous --strip-sections run
produces identical output, simplifying the test.
Reviewed by: rupprecht, grimar
Differential Revision: https://reviews.llvm.org/D59989
llvm-svn: 357475
Summary:
BTW, STLExtras.h provides llvm::size() which is similar to std::size()
for random access iterators. However, if we prefer qualified
llvm::size(), the member function .size() will be more convenient.
Reviewers: jhenderson, jakehehrlich, rupprecht, grimar, alexshap, espindola
Reviewed By: grimar
Subscribers: emaste, arichardson, jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60028
llvm-svn: 357347
This is a fix for https://bugs.llvm.org/show_bug.cgi?id=40007.
Idea is to swap the order of stripping. So that we strip sections before
symbols what allows us to strip relocation sections without emitting
the error about relative symbols.
Differential revision: https://reviews.llvm.org/D59763
llvm-svn: 357017
llvm-objcopy previously knew nothing about data in segments that wasn't
covered by section headers, meaning that it wrote zeroes instead of what
was there. As it is possible for this data to be useful to the loader,
this patch causes llvm-objcopy to start preserving this data. Data in
sections that are explicitly removed continues to be written as zeroes.
This fixes https://bugs.llvm.org/show_bug.cgi?id=41005.
Reviewed by: jakehehrlich, rupprecht
Differential Revision: https://reviews.llvm.org/D59483
llvm-svn: 356919
The idea of the patch is about to move out the code to a new
helper static functions (to reduce the size of 'handleArgs' and to
isolate the parts of it's logic).
Differential revision: https://reviews.llvm.org/D59762
llvm-svn: 356889
Fix: r356853 + set AddressAlign to 4 in
Inputs/compress-debug-sections.yaml for the new group section introduced.
Original commit message:
Currently, llvm-objcopy incorrectly handles compression and decompression of the
sections from COMDAT groups, because we do not implement the
replaceSectionReferences for this type of the sections.
The patch does that.
Differential revision: https://reviews.llvm.org/D59638
llvm-svn: 356856
This patch fixes the reason of ubsan failure (UB detected)
happened after landing the D59638 (I had to revert it).
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-ubsan/builds/11760/steps/check-llvm%20ubsan/logs/stdio)
Problem is the following. Our implementation of GroupSection assumes that
its address is 4 bytes aligned when writes it:
template <class ELFT>
void ELFSectionWriter<ELFT>::visit(const GroupSection &Sec) {
ELF::Elf32_Word *Buf =
reinterpret_cast<ELF::Elf32_Word *>(Out.getBufferStart() + Sec.Offset);
...
But the test case for D59638 did not set AddressAlign in YAML. So address was
not 4 bytes aligned since Sec.Offset was odd. That triggered the issue.
This patch teaches llvm-objcopy to report an error for such sections (which should
not met in reality), what is better than having UB.
Differential revision: https://reviews.llvm.org/D59695
llvm-svn: 356853
Currently, llvm-objcopy incorrectly handles compression and decompression of the
sections from COMDAT groups, because we do not implement the
replaceSectionReferences for this type of the sections.
The patch does that.
Differential revision: https://reviews.llvm.org/D59638
llvm-svn: 356738
GNU objcopy can support output formats like elf32-i386-freebsd and
elf64-x86-64-freebsd. The only difference from their regular non-freebsd
counterparts that I have observed is that the freebsd versions set the
OS/ABI field to ELFOSABI_FREEBSD. This patch sets the OS/ABI field
according based on the format whenever --output-format is specified.
Reviewed by: rupprecht, grimar
Differential Revision: https://reviews.llvm.org/D59645
llvm-svn: 356737
If the compression was used and we had a symbol not involved in relocation,
we never updated its section and it was silently removed from the output.
Differential revision: https://reviews.llvm.org/D59542
llvm-svn: 356554
This change makes linking into .build-id atomic and safe to use.
Some users under particular workflows are reporting that this races
more than half the time under particular conditions.
llvm-svn: 356404
This fixes the https://bugs.llvm.org/show_bug.cgi?id=40980.
Previously if string optimization occurred as a result of
StringTableBuilder's finalize() method, the size wasn't updated.
This hopefully also makes the interaction between sections during finalization
processes a bit more clear.
Differential revision: https://reviews.llvm.org/D59488
llvm-svn: 356371
For ELF, we accept but ignore --only-keep-debug. Do the same for llvm-strip.
COFF does implement this, so update the test that it is supported.
llvm-svn: 356207
This patch changes llvm-objcopy's behaviour to not strip sections that
are in segments, if they otherwise would be due to a stripping operation
(--strip-all, --strip-sections, --strip-non-alloc). This preserves the
segment contents. It does not change the behaviour of --strip-all-gnu
(although we could choose to do so), because GNU objcopy's behaviour in
this case seems to be to strip the section, nor does it prevent removing
of sections in segments with --remove-section (if a user REALLY wants to
remove a section, we should probably let them, although I could be
persuaded that warning might be appropriate). Tests have been added to
show this latter behaviour.
This fixes https://bugs.llvm.org/show_bug.cgi?id=41006.
Reviewed by: grimar, rupprecht, jakehehrlich
Differential Revision: https://reviews.llvm.org/D59293
This is a reland of r356129, attempting to fix greendragon failures
due to a suspected compatibility issue with od on the greendragon bots
versus other versions.
llvm-svn: 356136
This patch changes llvm-objcopy's behaviour to not strip sections that
are in segments, if they otherwise would be due to a stripping operation
(--strip-all, --strip-sections, --strip-non-alloc). This preserves the
segment contents. It does not change the behaviour of --strip-all-gnu
(although we could choose to do so), because GNU objcopy's behaviour in
this case seems to be to strip the section, nor does it prevent removing
of sections in segments with --remove-section (if a user REALLY wants to
remove a section, we should probably let them, although I could be
persuaded that warning might be appropriate). Tests have been added to
show this latter behaviour.
This fixes https://bugs.llvm.org/show_bug.cgi?id=41006.
Reviewed by: grimar, rupprecht, jakehehrlich
Differential Revision: https://reviews.llvm.org/D59293
llvm-svn: 356129
error() was previously cleaned up from CopyConfig, but new uses were introduced.
This also tweaks the error message for --add-symbol to report all invalid flags.
llvm-svn: 356105
When --compress-debug-sections is given,
llvm-objcopy removes the uncompressed sections and adds compressed to the section list.
This makes all the pointers to old sections to be outdated.
Currently, code already has logic for replacing the target sections of the relocation
sections. But we also have to update the relocations by themselves.
This fixes https://bugs.llvm.org/show_bug.cgi?id=40885.
Differential revision: https://reviews.llvm.org/D58960
llvm-svn: 355821
We should create CompressedSection only if the section has SHF_COMPRESSED flag
or it's name starts from '.zdebug'.
Currently, we create it if section's data starts from ZLIB signature.
Differential revision: https://reviews.llvm.org/D59018
llvm-svn: 355501
When --compress-debug-sections is given, llvm-objcopy do not compress
sections that have "ZLIB" header in data. Normally this signature is used
in zlib-gnu compression format. But if zlib-gnu used then the name of the compressed
section should start from .z* (e.g .zdebug_info). If it does not, then it is not
a zlib-gnu format and section should be treated as a normal uncompressed section.
Differential revision: https://reviews.llvm.org/D58908
llvm-svn: 355399
If zlib is not available, and --compress-debug-sections is passed,
we want to report an error. Currently, it is only reported for
--compress_debug_sections= form of the option.
Fixes the https://bugs.llvm.org/show_bug.cgi?id=40886.
I do not think there is a way to write a test for this.
Differential revision: https://reviews.llvm.org/D58909
llvm-svn: 355391
Summary:
This removes calls to `error()`/`reportError()` in the main driver (llvm-objcopy.cpp) as well as the associated argv-parsing (CopyConfig.cpp). `logAllUnhandledErrors()` is now the main way to print errors.
There are still a few uses from within the per-arch drivers, so we can't delete them yet... but almost!
Reviewers: jhenderson, alexshap, espindola
Reviewed By: jhenderson
Subscribers: emaste, arichardson, jakehehrlich, jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D58316
llvm-svn: 354600
Summary:
Removing a large number of sections from a file with a lot of symbols can have abysmal (i.e. O(n^2)) performance, e.g. when running `--only-section` to extract one section out of a large file.
This comes from iterating over all symbols in the symbol table each time we remove a section, to remove references to the section we just removed.
Instead, do just one pass of symbol removal by passing a hash set of all the sections we'd like to remove references to.
This fixes a regression when running llvm-objcopy -j <one section> on an object file with many sections and symbols -- on my machine, running `objcopy -j .keep_me huge-input.o /tmp/foo.o` takes .3s with GNU objcopy, 1.3s with an updated llvm-objcopy, and 7+ minutes with llvm-objcopy prior to this patch.
Reviewers: MaskRay, jhenderson, jakehehrlich, alexshap, espindola
Reviewed By: MaskRay, jhenderson
Subscribers: echristo, emaste, arichardson, mgrang, jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D58296
llvm-svn: 354597
A fallible iterator is one whose increment or decrement operations may fail.
This would usually be supported by replacing the ++ and -- operators with
methods that return error:
class MyFallibleIterator {
public:
// ...
Error inc();
Errro dec();
// ...
};
The downside of this style is that it no longer conforms to the C++ iterator
concept, and can not make use of standard algorithms and features such as
range-based for loops.
The fallible_iterator wrapper takes an iterator written in the style above
and adapts it to (mostly) conform with the C++ iterator concept. It does this
by providing standard ++ and -- operator implementations, returning any errors
generated via a side channel (an Error reference passed into the wrapper at
construction time), and immediately jumping the iterator to a known 'end'
value upon error. It also marks the Error as checked any time an iterator is
compared with a known end value and found to be inequal, allowing early exit
from loops without redundant error checking*.
Usage looks like:
MyFallibleIterator I = ..., E = ...;
Error Err = Error::success();
for (auto &Elem : make_fallible_range(I, E, Err)) {
// Loop body is only entered when safe.
// Early exits from loop body permitted without checking Err.
if (SomeCondition)
return;
}
if (Err)
// Handle error.
* Since failure causes a fallible iterator to jump to end, testing that a
fallible iterator is not an end value implicitly verifies that the error is a
success value, and so is equivalent to an error check.
Reviewers: dblaikie, rupprecht
Subscribers: mgorny, dexonsmith, kristina, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D57618
llvm-svn: 353237
Summary: Use StringSaver/BumpPtrAlloc when parsing lines from --keep-global-symbols files. This allows us to consistently use StringRef for driver options, which avoids copying the full strings for each object copied, as well as simplifies part of D57517.
Reviewers: jhenderson, evgeny777, alexshap
Subscribers: jakehehrlich
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D57617
llvm-svn: 353068
This diff implements first bits for copying (without modification) MachO object files.
Test plan: make check-all
Differential revision: https://reviews.llvm.org/D54674
llvm-svn: 352944
Summary:
Replace some reportError() calls with error propagation that was missed from rL352625.
Note this also adds an error check during Archive iteration that was being hidden by a different error check before:
```
for (const Archive::Child &Child : Ar.children(Err)) {
Expected<std::unique_ptr<Binary>> ChildOrErr = Child.getAsBinary();
if (!ChildOrErr)
// This aborts, so Err is never checked
reportError(Ar.getFileName(), ChildOrErr.takeError());
```
Err is being checked after the loop, so during happy runs, everything is fine. But when reportError is changed to return the error instead of aborting, the fact that Err is never checked is now noticed in tests that trigger an error during the loop.
Reviewers: jhenderson, dblaikie, alexshap
Reviewed By: dblaikie
Subscribers: llvm-commits, lhames, jakehehrlich
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D57462
llvm-svn: 352888
Summary:
Include the symbol being defined in the list of requirements for using --localize-symbol.
This is used, for example, when someone is depending on two different projects that have the same (or close enough) method defined in each library, and using "-L sym" for a conflicting symbol in one of the libraries so that the definition from the other one is used. However, the library may have internal references to the symbol, which cause program crashes when those are used, i.e.:
```
$ cat foo.c
int foo() { return 5; }
$ cat bar.c
int foo();
int bar() { return 2 * foo(); }
$ cat foo2.c
int foo() { /* Safer implementation */ return 42; }
$ cat main.c
int bar();
int main() {
__builtin_printf("bar = %d\n", bar());
return 0;
}
$ ar rcs libfoo.a foo.o bar.o
$ ar rcs libfoo2.a foo2.o
# Picks the wrong foo() impl
$ clang main.o -lfoo -lfoo2 -L. -o main
# Picks the right foo() impl
$ objcopy -L foo libfoo.a && clang main.o -lfoo -lfoo2 -L. -o main
# Links somehow, but crashes at runtime
$ llvm-objcopy -L foo libfoo.a && clang main.o -lfoo -lfoo2 -L. -o main
```
Reviewers: jhenderson, alexshap, jakehehrlich, espindola
Subscribers: emaste, arichardson
Differential Revision: https://reviews.llvm.org/D57417
llvm-svn: 352767
Summary:
This adds support for the --discard-locals flag, which acts similarly to --discard-all, except it only applies to compiler-generated symbols (i.e. symbols starting with `.L` in ELF).
I am not sure about COFF local symbols: those appear to also use `.L` in most cases, but also use just `L` in other cases, so for now I am just leaving it unimplemented there.
Fixes PR36160
Reviewers: jhenderson, alexshap, jakehehrlich, mstorsjo, espindola
Reviewed By: jhenderson
Subscribers: llvm-commits, emaste, arichardson
Differential Revision: https://reviews.llvm.org/D57248
llvm-svn: 352626
Summary: Do some more error cleanup, removing some dependencies from llvm-objcopy's error/reportError in [ELF/COFF]Objcopy methods.
Reviewers: jhenderson, alexshap, jakehehrlich, mstorsjo, espindola
Subscribers: emaste, arichardson
Differential Revision: https://reviews.llvm.org/D57423
llvm-svn: 352625
Summary:
--set-section-flags is used to change the section flags (e.g. SHF_ALLOC) for given sections. The flags allowed are the same from the existing --rename-section=.old=.new[,flags] feature.
Additionally, make sure that --set-section-flag cannot be used with --rename-section (either the source or destination), since --rename-section accepts flags. This avoids ambiguity for something like "--rename-section=.foo=.bar,alloc --set-section-flag=.bar,code".
Reviewers: jhenderson, jakehehrlich, alexshap, espindola
Reviewed By: jhenderson, jakehehrlich
Subscribers: llvm-commits, emaste, arichardson
Differential Revision: https://reviews.llvm.org/D57198
llvm-svn: 352505
Summary: When using llvm-objcopy -O binary and the resulting file will be empty (e.g. removing the only section that would be written, or using --only-keep with a section that doesn't exist/isn't SHF_ALLOC), we crash because FileOutputBuffer expects Size > 0. Add a regression test, and change Buffer to open/truncate the output file in this case.
Reviewers: alexshap, jhenderson, jakehehrlich, espindola
Reviewed By: alexshap, jhenderson
Subscribers: jfb, llvm-commits, emaste, arichardson
Differential Revision: https://reviews.llvm.org/D56806
llvm-svn: 352371
This change adds an option -g to llvm-objcopy which is an alias for the existing option --strip-debug.
This fixes PR40003.
Reviewed by: alexshap
Differential Revision: https://reviews.llvm.org/D57217
llvm-svn: 352182
The aux symbols were stored in an opaque std::vector<uint8_t>,
with contents interpreted according to the rest of the symbol.
All aux symbol types but one fit in 18 bytes (sizeof(coff_symbol16)),
and if written to a bigobj, two extra padding bytes are written (as
sizeof(coff_symbol32) is 20). In the storage agnostic intermediate
representation, store the aux symbols as a series of coff_symbol16
sized opaque blobs. (In practice, all such aux symbols only consist
of one aux symbol, so this is more flexible than what reality needs.)
The special case is the file aux symbols, which are written in
potentially more than one aux symbol slot, without any padding,
as one single long string. This can't be stored in the same opaque
vector of fixed sized aux symbol entries. The file aux symbols will
occupy a different number of aux symbol slots depending on the type
of output object file. As nothing in the intermediate process needs
to have accurate raw symbol indices, updating that is moved into the
writer class.
Differential Revision: https://reviews.llvm.org/D57009
llvm-svn: 351947
This was reverted since it broke a couple buildbots. The reason
for the breakage is not yet known, but this time, the test has
got more diagnostics added, to hopefully allow figuring out
what goes wrong.
Differential Revision: https://reviews.llvm.org/D57007
llvm-svn: 351931
Summary:
This patch changes a few methods to return Error instead of manually calling error/reportError to abort. This will make it easier to extract into a library.
Note that error() takes just a string (this patch also adds an overload that takes an Error), while reportError() takes string + [error/code]. To help unify things, use FileError to associate a given filename with an error. Note that this takes some special care (for now), e.g. calling reportError(FileName, <something that could be FileError>) will duplicate the filename. The goal is to eventually remove reportError() and have every error associated with a file to be a FileError, and just one error handling block at the tool level.
This change was suggested in D56806. I took it a little further than suggested, but completely fixing llvm-objcopy will take a couple more patches. If this approach looks good, I'll commit this and apply similar patche(s) for the rest.
This change is NFC in terms of non-error related code, although the error message changes in one context.
Reviewers: alexshap, jhenderson, jakehehrlich, mstorsjo, espindola
Reviewed By: alexshap, jhenderson
Subscribers: llvm-commits, emaste, arichardson
Differential Revision: https://reviews.llvm.org/D56930
llvm-svn: 351896
This was requested in the review of D57006.
Also add missing quotes around symbol names in error messages.
Differential Revision: https://reviews.llvm.org/D57014
llvm-svn: 351799
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
Fix llvm-objcopy to add .note sections as SHT_NOTEs. GNU objcopy overrides section flags for special sections. For `.note` sections (with the exception of `.note.GNU-stack`), SHT_NOTE is used.
Many other sections are special cased by libbfd, but `.note` is the only special section I can seem to find being used with objcopy --add-section.
See `.note` in context of the full list of special sections here: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=bfd/elf.c;h=eb3e1828e9c651678b95a1dcbc3b124783d1d2be;hb=HEAD#l2675
Reviewers: jhenderson, alexshap, jakehehrlich, espindola
Reviewed By: jhenderson
Subscribers: emaste, arichardson, llvm-commits
Differential Revision: https://reviews.llvm.org/D56570
llvm-svn: 351204
These were copied as part of the original design from the ELF
backend, but aren't necessary at the moment.
Differential Revision: https://reviews.llvm.org/D56431
llvm-svn: 350996
Previously, this was broken - by setting PointerToSymbolTable to zero
but still actually writing the string table length, the object file
header was corrupted.
Differential Revision: https://reviews.llvm.org/D56584
llvm-svn: 350926
Summary:
The -O flag is currently being mostly ignored; it's only checked whether or not the output format is "binary". This adds support for a few formats (e.g. elf64-x86-64), so that when specified, the output can change between 32/64 bit and sizes/alignments are updated accordingly.
This fixes PR39135
Reviewers: jakehehrlich, jhenderson, alexshap, espindola
Reviewed By: jhenderson
Subscribers: emaste, arichardson, llvm-commits
Differential Revision: https://reviews.llvm.org/D53667
llvm-svn: 350541
Summary:
Fix EntrySize, Size, and Align before doing layout calculation.
As a side cleanup, this removes a dependence on sizeof(Elf_Sym) within BinaryReader, so we can untemplatize that.
This unblocks a cleaner implementation of handling the -O<format> flag. See D53667 for a previous attempt. Actual implementation of the -O<format> flag will come in an upcoming commit, this is largely a NFC (although not _totally_ one, because alignment on binary input was actually wrong before).
Reviewers: jakehehrlich, jhenderson, alexshap, espindola
Reviewed By: jhenderson
Subscribers: emaste, arichardson, llvm-commits
Differential Revision: https://reviews.llvm.org/D56211
llvm-svn: 350336
This fixes all cases of errors in asan+ubsan builds.
Also use std::copy instead of if+memcpy in the previously updated spot,
for consistency.
llvm-svn: 349826
