Commit Graph

144 Commits

Author SHA1 Message Date
Pavel Labath
cf82cc3d54 dotest: consistently call finalize_build_dictionary in debug info variants
dwarf&dwo versions were doing it, but gmodules and dsym weren't. All
this function does right now is pass OS=Android to make when targeting
android. This enables us to run dotest without manually passing --env
OS=Android.

llvm-svn: 317130
2017-11-01 22:01:03 +00:00
Davide Italiano
9e03e17df1 [lldbtest] Simplify removing an unneeded else. NFCI.
llvm-svn: 316355
2017-10-23 17:51:22 +00:00
Jason Molenda
8652b249e6 Initial patchset to get the testsuite running against armv7 and arm64 iOS devices.
Normal customer devices won't be able to run these tests, we're hoping to get
a public facing bot set up at some point.  Both devices pass the testsuite without
any errors or failures.

I have seen some instability with the armv7 test runs, I may submit additional patches
to address this.  arm64 looks good.

I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same.  I know it's a rather large patch.

One change I had to make specifically for iOS devices was that debugserver can't 
create files.  There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file.  They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.

llvm-svn: 314132
2017-09-25 18:19:39 +00:00
Chris Bieneman
9568a5102e Revert "Initial patchset to get the testsuite running against armv7 and arm64 iOS devices. Normal customer devices won't be able to run these devices, we're hoping to get a public facing bot set up at some point. Both devices pass the testsuite without any errors or failures."
This patch has been causing LLDB test failures on ObjC tests. A test log
may still be available here:

http://lab.llvm.org:8080/green/view/LLDB/job/lldb/1650/

This reverts commit r314038.

llvm-svn: 314122
2017-09-25 17:31:40 +00:00
Jason Molenda
0187a8f6f9 Initial patchset to get the testsuite running against armv7 and arm64 iOS devices.
Normal customer devices won't be able to run these devices, we're hoping to get
a public facing bot set up at some point.  Both devices pass the testsuite without
any errors or failures.

I have seen some instability with the armv7 test runs, I may submit additional patches
to address this.  arm64 looks good.

I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same.  I know it's a rather large patch.

One change I had to make specifically for iOS devices was that debugserver can't 
create files.  There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file.  They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.

llvm-svn: 314038
2017-09-22 22:34:53 +00:00
Jason Molenda
2d5d71c061 Revert this patch; I was emailing with Eugene and they have some other changes going
in today and don't want the two changes to confuse the situation with the build bots.
I'll commit tomorrow once they're known good.

llvm-svn: 313934
2017-09-21 23:02:56 +00:00
Jason Molenda
182a8083c1 Initial patchset to get the testsuite running against armv7 and arm64 iOS devices.
Normal customer devices won't be able to run these devices, we're hoping to get
a public facing bot set up at some point.

There will be some smaller follow-on patches.  The changes to tools/lldb-server are
verbose and I'm not thrilled with having to skip all of these tests manually.
There are a few places where I'm making the assumption that "armv7", "armv7k", "arm64"
means it's an ios device, and I need to review & clean these up with an OS check
as well.  (Android will show up as "arm" and "aarch64" so by pure luck they shouldn't
cause problems, but it's not an assumption I want to rely on).

I'll be watching the bots for the rest of today; if any problems are introduced by
this patch I'll revert it - if anyone sees a problem with their bot that I don't
see, please do the same.  I know it's a rather large patch.

One change I had to make specifically for iOS devices was that debugserver can't 
create files.  There were several tests that launch the inferior process redirecting
its output to a file, then they retrieve the file.  They were not trying to test
file redirection in these tests, so I rewrote those to write their output to a file
directly.

llvm-svn: 313932
2017-09-21 23:00:19 +00:00
Pavel Labath
1b8fe5ec8d Make TestConflictingSymbol run on non-darwin targets
For remote targets we need to call registerSharedLibrariesWithTarget to
make sure they are installed alongside main executable. This also
required a small fixup in the the mentioned function as in this case
"One" was both a directory name and a library name template. I fixed it
to make sure it checks that the string refers to a file before it
assumed it was a full library path.

llvm-svn: 303248
2017-05-17 11:47:44 +00:00
Pavel Labath
6b42b3b7a3 Fix remote test suite directory creation
r298203 make SBPlatform::MakeDirectory less recursive, which breaks the
test suite creation of test directory hierarchy creation on the remote
target. Since the function was never fully recursive, and the name does
not imply recursiveness, I fix the problem by modifying the test runner
to do the recursion manually.

I also make the runner complain more loudly when it fails to create the
directory -- previously it just printed the error to stdout and caused
most of the tests to hang, which is not very helpful in diagnosing the
problem.

llvm-svn: 298261
2017-03-20 16:07:17 +00:00
Pavel Labath
6de25ec61a dotest.py: remove the ability to specify different architectures/compilers in a single invocation
Summary:
This has been broken at least since the new test result framework was
added, which was over a year ago. It looks like nobody has missed it
since.

Removing this makes the gmodules handling code saner, as it already did
not know how to handle the multiple-compilers case.

My motivation for this is libc++ data formatters support on android -- I
am trying make a central way of determining whether libc++ tests can be
run, and without this, I would have to resort to similar hacks as the
gmodules code.

Reviewers: jingham, zturner

Subscribers: danalbert, tfiala, lldb-commits

Differential Revision: https://reviews.llvm.org/D30779

llvm-svn: 297811
2017-03-15 08:51:59 +00:00
Pavel Labath
4438bf7498 test: shorten test trace file names
Make sure we don't generate extremely long file names for test trace log
file, as this can cause path-too-long errors. As the compilers in the
android ndk are deeply nested, it's very easy to trigger these.

I chose to output at most 4 path components -- this should keep the full
path for common cases like /usr/bin/gcc with room to spare, and should
be enough to uniquely identify the compiler for more deeply nested
cases.

llvm-svn: 296870
2017-03-03 13:49:38 +00:00
Nitesh Jain
8493e6a889 [LLDB][MIPS] Fix TestMiExec and TestMiData failures
Subscribers: jaydeep, bhushan, lldb-commits, slthakur
llvm-svn: 294418
2017-02-08 07:42:56 +00:00
Zachary Turner
c156427ded Don't allow direct access to StreamString's internal buffer.
This is a large API change that removes the two functions from
StreamString that return a std::string& and a const std::string&,
and instead provide one function which returns a StringRef.

Direct access to the underlying buffer violates the concept of
a "stream" which is intended to provide forward only access,
and makes porting to llvm::raw_ostream more difficult in the
future.

Differential Revision: https://reviews.llvm.org/D26698

llvm-svn: 287152
2016-11-16 21:15:24 +00:00
Chris Bieneman
b6f833793d [Test Suite] Properly quote python string
Oops!

llvm-svn: 286258
2016-11-08 18:14:42 +00:00
Chris Bieneman
4b8dbcb74c [Test Suite] Attempt to fix issue zturner reported to me
Not sure why this didn't explode more massively, but this should fix the issue with the non-framework tests.

llvm-svn: 286254
2016-11-08 17:53:18 +00:00
Chris Bieneman
f8ac2da114 [Test Suite] Pull generateSource into lldbtest
Summary:
Convert tests using LLDB headers to use generateSource to put the right include paths in place regardless of whether or not you're building a framework.

This also abstracted generateSource out of TestPublicAPIHeaders.py into lldbtest.py.

Reviewers: tfiala, zturner

Subscribers: lldb-commits

Differential Revision: https://reviews.llvm.org/D25887

llvm-svn: 285542
2016-10-31 04:48:19 +00:00
Chris Bieneman
bd6d69987c [Test Suite] Properly respect --framework option
Summary:
dotest.py has a framework option that is not respected. This patch makes the framework path properly configurable via the --framework option.

This patch also adds a function to the lldbtest.Base class named "hasDarwinFramework" which allows us to not rely on the host platform to determine if a framework is present. If running on Darwin, and not building a framework, this will follow the *nix code paths which are appropriate for Darwin.

Reviewers: tfiala

Subscribers: lldb-commits, mgorny

Differential Revision: https://reviews.llvm.org/D25886

llvm-svn: 285541
2016-10-31 04:48:10 +00:00
Chris Bieneman
614dc150ec Revert "[Test Suite] Pull generateSource into lldbtest"
This reverts commit r285357.

I committed this patch accidentally out of order. Will recommit when the change this depends on is landed.

llvm-svn: 285361
2016-10-27 23:18:52 +00:00
Chris Bieneman
f817bae161 [Test Suite] Pull generateSource into lldbtest
Summary:
Convert tests using LLDB headers to use generateSource to put the right include paths in place regardless of whether or not you're building a framework.

This also abstracted generateSource out of TestPublicAPIHeaders.py into lldbtest.py.

Reviewers: tfiala, zturner

Subscribers: lldb-commits

Differential Revision: https://reviews.llvm.org/D25887

llvm-svn: 285357
2016-10-27 22:52:32 +00:00
Chris Bieneman
7ba5581b84 [Test Suite] Allow overriding codesign identity
Summary: Not everyone names their code sign identity "lldb_codesign", so it is nice to allow this to be overridden.

Reviewers: zturner, tfiala

Subscribers: labath, mgorny, lldb-commits

Differential Revision: https://reviews.llvm.org/D25714

llvm-svn: 284893
2016-10-21 22:13:55 +00:00
Todd Fiala
9666ba7526 add stop column highlighting support
This change introduces optional marking of the column within a source
line where a thread is stopped.  This marking will show up when the
source code for a thread stop is displayed, when the debug info
knows the column information, and if the optional column marking is
enabled.

There are two separate methods for handling the marking of the stop
column:

* via ANSI terminal codes, which are added inline to the source line
  display.  The default ANSI mark-up is to underline the column.

* via a pure text-based caret that is added in the appropriate column
  in a newly-inserted blank line underneath the source line in
  question.

There are some new options that control how this all works.

* settings set stop-show-column

  This takes one of 4 values:

  * ansi-or-caret: use the ANSI terminal code mechanism if LLDB
    is running with color enabled; if not, use the caret-based,
    pure text method (see the "caret" mode below).

  * ansi: only use the ANSI terminal code mechanism to highlight
    the stop line.  If LLDB is running with color disabled, no
    stop column marking will occur.

  * caret: only use the pure text caret method, which introduces
    a newly-inserted line underneath the current line, where
    the only character in the new line is a caret that highlights
    the stop column in question.

  * none: no stop column marking will be attempted.

* settings set stop-show-column-ansi-prefix

  This is a text format that indicates the ANSI formatting
  code to insert into the stream immediately preceding the
  column where the stop column character will be marked up.
  It defaults to ${ansi.underline}; however, it can contain
  any valid LLDB format codes, e.g.

      ${ansi.fg.red}${ansi.bold}${ansi.underline}

* settings set stop-show-column-ansi-suffix

  This is the text format that specifies the ANSI terminal
  codes to end the markup that was started with the prefix
  described above.  It defaults to: ${ansi.normal}.  This
  should be sufficient for the common cases.

Significant leg-work was done by Adrian Prantl.  (Thanks, Adrian!)

differential review: https://reviews.llvm.org/D20835

reviewers: clayborg, jingham
llvm-svn: 282105
2016-09-21 20:13:14 +00:00
Kate Stone
b9c1b51e45 *** This commit represents a complete reformatting of the LLDB source code
*** to conform to clang-format’s LLVM style.  This kind of mass change has
*** two obvious implications:

Firstly, merging this particular commit into a downstream fork may be a huge
effort.  Alternatively, it may be worth merging all changes up to this commit,
performing the same reformatting operation locally, and then discarding the
merge for this particular commit.  The commands used to accomplish this
reformatting were as follows (with current working directory as the root of
the repository):

    find . \( -iname "*.c" -or -iname "*.cpp" -or -iname "*.h" -or -iname "*.mm" \) -exec clang-format -i {} +
    find . -iname "*.py" -exec autopep8 --in-place --aggressive --aggressive {} + ;

The version of clang-format used was 3.9.0, and autopep8 was 1.2.4.

Secondly, “blame” style tools will generally point to this commit instead of
a meaningful prior commit.  There are alternatives available that will attempt
to look through this change and find the appropriate prior commit.  YMMV.

llvm-svn: 280751
2016-09-06 20:57:50 +00:00
Pavel Labath
7cf00d191a Enable test log collection from remote debug servers (take 2)
Summary:
This is a slightly reworked version of D16322, which I had reverted because it did not do what it
advertised. Differences from the previous version are:
- moved the code for cleaning up the remote working dir to a later point as it was removing the
  log file before we could get to it.
- specialised log downloading code for gdb-remote tests is not needed, as this will cover that
  use case as well.

Reviewers: tfiala

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D21898

llvm-svn: 274491
2016-07-04 09:59:45 +00:00
Todd Fiala
94eb010fe9 Add "-gmodules" support to the test suite.
This change adds the capability of building test inferiors
with the -gmodules flag to enable module debug info support.
Windows is excluded per @zturner.

Reviewers: granata.enrico, aprantl, zturner, labath

Subscribers: zturner, labath, lldb-commits

Differential Revision: http://reviews.llvm.org/D19998

llvm-svn: 270848
2016-05-26 13:57:03 +00:00
Zachary Turner
8d4d151bb2 Allow custom formatting of session log file names.
Differential Revision: http://reviews.llvm.org/D20306

llvm-svn: 269793
2016-05-17 18:02:34 +00:00
Todd Fiala
4728cf7e85 surface build error content through test event system
Summary:
print build errors nicely in test output

This test infrastructure change adds a new Python exception
for test subject builds that fail.  The output of the build
command is captured and propagated to both the textual test
output display code and to the test event system.

The ResultsFormatter objects have been modified to do something
more useful with this information.  The xUnit formatter
now replaces the non-informative Python build error stacktrace
with the build error content.  The curses ResultsFormatter
prints a 'B' for build errors rather than 'E'.

The xUnit output, in particular, makes it much easier for
developers to track down test subject build errors that cause
test failures when reports come in from CI.

Reviewers: granata.enrico

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D20252

llvm-svn: 269525
2016-05-14 00:42:30 +00:00
Todd Fiala
49d3c15c3e test infra: move test event-related handling into its own package
This change moves all the test event handling and its related
ResultsFormatter classes out of the packages/Python/lldbsuite/test dir
into a packages/Python/lldbsuite/test_event package. Formatters are
moved into a sub-package under that.

I am limiting the scope of this change to just the motion and a few
minor issues caught by a static Python checker (e.g. removing unused
import statements).

This is a pre-step for adding package-level tests to the test event
system. I also intend to simplify test event results formatter selection
after I make sure this doesn't break anybody.

See:
http://reviews.llvm.org/D19288

Reviewed by:
Pavel Labath

llvm-svn: 266885
2016-04-20 16:27:27 +00:00
Adrian McCarthy
121571b7ce Retry deletion of temporary files to avoid race conditions on Windows.
Differential Revision: http://reviews.llvm.org/D18912

llvm-svn: 265948
2016-04-11 15:21:01 +00:00
Tamas Berghammer
19fc1d4e3c [NFC] Cleanup the code used to run shell commands from tests
Previously we had 3 different method to run shell commands on the
target and 4 copy of code waiting until a given file appears on the
target device (used for syncronization). This CL merges these methods
to 1 run_platform_command and 1 wait_for_file_on_target functions
located in some utility classes.

Differential revision: http://reviews.llvm.org/D18789

llvm-svn: 265398
2016-04-05 14:08:18 +00:00
Pavel Labath
3f081c8210 Don't vary debug info for lldb-server tests
Summary:
Debug info is used only by the client and lldb-server tests do not even have the client component
running, as they communicate with the server directly. Therefore, running the tests for each
debug info type is unnecessarry.

This adds general ability to mark a test class as not dependent on debug info, and marks all
lldb-server tests as such.

Reviewers: tberghammer, tfiala

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D18598

llvm-svn: 265017
2016-03-31 14:22:52 +00:00
Sean Callanan
bbde9083ad Made self.expect() errors a little more readable in the testsuite.
self.expect() had two problems:

- If there was a substrs argument, then it overwrote the variable containing
  the command to run with the last substr.  That meant nonsense command text in
  testsuite errors.

- The actual output is not printed, which makes fixing testsuite failures a bit
  annoying (you end up having to use the -tv arguments to dotest).

This fixes both of these issues.  We could do even better, pretty-printing the
criteria for "correct" output, but this at least makes dealing with errors a bit
better.

llvm-svn: 262950
2016-03-08 18:58:48 +00:00
Siva Chandra
29d9bea93f Adjust for Python-3.
Summary:
This does not yet give us a clean testsuite run but it does help with:
1. Actually building on linux
2. Run the testsuite with over 70% tests passing on linux.

Reviewers: tfiala, labath, zturner

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D17182

llvm-svn: 260721
2016-02-12 20:30:47 +00:00
Zachary Turner
9a1a2946af Move the rest of the tests over to using the new decorator module.
llvm-svn: 259838
2016-02-04 23:04:17 +00:00
Zachary Turner
7a5382de82 Move some of the common decorators to decorators.py.
This doesn't attempt to move every decorator.  The reason for
this is that it requires touching every single test file to import
decorators.py.  I would like to do this in a followup patch, but
in the interest of keeping the patches as bite-sized as possible,
I've only attempted to move the underlying common decorators first.
A few tests call these directly, so those tests are updated as part
of this patch.

llvm-svn: 259807
2016-02-04 18:03:01 +00:00
Zachary Turner
62d3a6570a Move some android platform functions to lldbplatformutil.
My eventual goal is to move all of the test decorators to their
own module such as `decorators.py`.  But some of the decorators
use existing functions in `lldbtest.py` and conceptually the
functions are probably more appropriately placed in lldbplatformutil.
Moreover, lldbtest.py is a huge file with a ton of random utility
functions scattered around, so this patch also workds toward the
goal of reducing the footprint of this one module to a more
reasonable size.

So this patch moves some of them over to lldbplatformutil with the
eventual goal of moving decorators over to their own module.

Reviewed By: Tamas Berghammer, Pavel Labath
Differential Revision: http://reviews.llvm.org/D16830

llvm-svn: 259680
2016-02-03 19:12:30 +00:00
Pavel Labath
5f496fb941 Remove skipUnlessListedRemote test decorator
This decorator was used in only one test, and it's behaviour was quite complicated. It skipped
if:
- test was remote
- platform was *not* android

I am not aware of anyone running tests with this configuration (and even then, I am not aware of
a reason why the test should not pass), but if TestLoadUnload starts breaking for you after this
commit, please disable the test with
@expectedFailureAll(remote=True, oslist=[YOUR_PLATFORM])

llvm-svn: 259642
2016-02-03 11:51:25 +00:00
Zachary Turner
c67466054f Don't return a tuple from the skip test function.
Previously we were returning a tuple of (bool, skip_reason) from
the tuple function.  This makes for some awkward code, especially
since a value of True for the first argument implies that the
second argument is None, and a value of False implies that the
second argument is not None.  So it was basically redundant, and
with this patch we simply return the skip reason or None directly.

llvm-svn: 259590
2016-02-02 22:22:53 +00:00
Zachary Turner
d6609d74a1 Re-write many skip decorators to use shared code.
This should be no functional change, just a refactoring of the
skip decorators to all centralize on a single function,
`skipTestIfFn` that does all the logic.  This allows easier
maintenance of the decorators and also centralizes all the
hard-to-understand logic in one place.

Reviewed by: Pavel Labath
Differential Revision: http://reviews.llvm.org/D16741

llvm-svn: 259543
2016-02-02 18:50:34 +00:00
Pavel Labath
e8baa4498d Fix compiler lookup when specified without path
r259433 introduced a regression, where if a compiler is specified without a path (e.g., CC=clang,
relying on the fact that clang is in $PATH), then the test suite would fail (at the compiler
version detection step) because realpath would interpret this as a path relative to cwd). The fix
is to perform the $PATH expansion (via `which`) before the realpath step.

llvm-svn: 259484
2016-02-02 09:49:37 +00:00
Zachary Turner
e5a7990dbe Always write the session file in UTF-8.
This patch attempts to solve the Python 2 / Python 3 incompatibilities by
introducing a new `encoded_file` abstraction that we use instead of
`io.open()`.  The problem with the builtin implementation of `io.open` is
that `read` and `write` accept and return `unicode` objects, which are not
always convenient to work with in Python 2.  We solve this by making
`encoded_file.open()` return the same object returned by `io.open()` but
with hooked `read()` and `write()` methods.  These hooked methods will
accept binary or text data, and conditionally convert what it gets to a
`unicode` object using the correct encoding.  When calling `read()` it
also does any conversion necessary to convert the output back into the
native `string` type of the running python version.

Differential Revision: http://reviews.llvm.org/D16736

llvm-svn: 259379
2016-02-01 18:12:59 +00:00
Zachary Turner
95b533fe4b Revert "Resubmit r258759 with proper unicode handling."
This reverts commit 2c79d60214e146b13b233392a859b4f79340e90e.

llvm-svn: 258978
2016-01-27 19:47:28 +00:00
Zachary Turner
671e6340da Resubmit r258759 with proper unicode handling.
Instead of opening the file in unicode mode, we need only encode
data which potentially has non-ASCII characters as UTF8 before
writing.  This should work across both Python versions, and is
also far simpler than anything else discussed.

llvm-svn: 258969
2016-01-27 19:00:51 +00:00
Zachary Turner
e89a66bd4a Fix some python 3 incompatibilities that went in overnight.
* basestring is not a thing anymore.  Must use `six.string_types`.
* Must use from __future__ import print_function in every new test
  file.

llvm-svn: 258967
2016-01-27 18:49:35 +00:00
Zachary Turner
7289e43831 Refactor some of the xfail / skip decorators to share logic.
Previously the logic of skipIf and expectedFailure were 99%
the same, but they took different sets of arguments since they
were maintained separately, and had slightly differences in
their behavior.  This makes everything consistent, there is now
only one real implementation, and the previous ones are changed
to use the single master implementation.

llvm-svn: 258966
2016-01-27 18:49:31 +00:00
Bhushan D. Attarde
df5f0b448c [LLDB][MIPS] A small fix in GetBreakableLoadAddress() for MIPS
SUMMARY:
    Get the load address for the address given by symbol and function.
    Earlier, this was done for function only, this patch does it for symbol too.
    This patch also adds TestAvoidBreakpointInDelaySlot.py to test this change.
    
    Reviewers: clayborg
    Subscribers: labath, zturner, mohit.bhakkad, sagar, jaydeep, lldb-commits
    Differential Revision: http://reviews.llvm.org/D16049

llvm-svn: 258919
2016-01-27 10:16:30 +00:00
Enrico Granata
dd54a3a887 Reverting r258759 as it is breaking the OSX build
llvm-svn: 258791
2016-01-26 04:53:10 +00:00
Zachary Turner
00b87282cd Write the session log file in UTF-8.
Previously we were writing in the default encoding, which depends
on the operating system and is not guaranteed to be unicode aware.
On Python 3, this would lead to a situation where writing unicode
text to the log file generates an exception.  The fix here is to
write session logs using the proper encoding, which incidentally
fixes another test, so xfail is removed from that.

llvm-svn: 258759
2016-01-26 00:59:42 +00:00
Zachary Turner
8012496a91 Decode files with UTF-8 in lldbutil.line_number.
Since Unicode support is different in Py2 and Py3, Py3 was throwing
exceptions about being unable to decode the file with the default
encoding.

llvm-svn: 258588
2016-01-22 23:54:49 +00:00
Pavel Labath
4b70eb7b2b Revert "Enable test log collection from remote debug servers"
Unfortunately, this turns out not to be working on the lldb-server tests, as there the server is
started in a different way. Since this was a bit of a hack to start with, I am removing it until
I can solve the problem more holistically.

llvm-svn: 258501
2016-01-22 14:50:29 +00:00
Pavel Labath
e3e6be2aab Enable test log collection from remote debug servers
Summary:
We already have the ability to collect the server logs when doing local debugging. This enables
the collection of remote logs as well. This relies on specifying a relative path "server.log" for
LLDB_DEBUGSERVER_LOG_FILE when starting remote platform. Since we always set the platform working
directory to a fresh folder to avoid conflicts, the actual file path will always be different and
we can pick the logs up from there.

Reviewers: tfiala

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D16322

llvm-svn: 258414
2016-01-21 17:54:14 +00:00
Pavel Labath
bace94eda8 Remove skipIfLinuxClang decorator
it isn't used in the code anymore, and we're trying to cut down on the decorators.

llvm-svn: 258049
2016-01-18 15:01:14 +00:00
Pavel Labath
4da73573ab Guard against application of an XFAIL decorator on a class
This does not work and causes the class to be silently skipped, which is a bad idea. This makes
sure it cannot happen accidentaly. I've played with the idea of actually making the decorator
work at class level, but it proved too magic to do at this moment.

llvm-svn: 258048
2016-01-18 14:45:35 +00:00
Adrian McCarthy
aa1f270151 Fix detection of teardown hooks that are already bound.
llvm-svn: 257829
2016-01-14 22:52:16 +00:00
Omair Javaid
baa07490bc Xfail some Arm-Linux specific failures
Updated expectedFailureLinux decorator to reflect architecture

Marked some triaged failures as xfails on arm with updated expectedFailureLinux decorator

Differential revision: http://reviews.llvm.org/D15893

llvm-svn: 257405
2016-01-11 22:52:18 +00:00
Pavel Labath
773e86f255 Remove old flaky test rerun logic
Summary:
This removes the old logic for rerunning flaky tests. The new test runners will take care of
rerunning failing tests.

Reviewers: tfiala

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15855

llvm-svn: 256824
2016-01-05 10:44:36 +00:00
Adrian McCarthy
46155dd7a9 Allow test decorators to use lists (and not_in(list)) for arches
Differential Revision: https://mail.google.com/mail/u/0/?zx=w4areffgjbgg#inbox/151cb6afe6169bb0

llvm-svn: 256283
2015-12-22 21:01:21 +00:00
Pavel Labath
25e241b006 [test] Add ability to expect timeouts
Summary:
This adds ability to mark test that do not complete due to hangs, crashes, etc., as "expected",
to avoid flagging the build red for a known problem. Functionally, this extends the scope of the
existing expectedFailureXXX decorators to cover these states as well. Once this is in, I will
start replacing the magic list of failing tests in dosep.py with our regular annotations which
should hopefully make code simpler.

Reviewers: tfiala

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15530

llvm-svn: 255763
2015-12-16 12:09:45 +00:00
Siva Chandra
a386358c36 Make few adjustments after r255542.
Reviewers: zturner

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15511

llvm-svn: 255584
2015-12-15 00:26:52 +00:00
Zachary Turner
8a927c4c10 Remove the multiplier loop.
This is leading to some kind of subtle issue related to local
functions and closures, so let's just go back to the old way for
now.

llvm-svn: 255567
2015-12-14 22:58:16 +00:00
Zachary Turner
e1eb5e39f5 Make skipIf decorator support not_in() functor.
llvm-svn: 255542
2015-12-14 21:26:49 +00:00
Zachary Turner
f098e4fb19 Make debug info specification use categories system.
Reviewed By: Tamas Berghammer, Pavel Labath
Differential Revision: http://reviews.llvm.org/D15428

llvm-svn: 255525
2015-12-14 18:49:16 +00:00
Pavel Labath
ffbf9e86b2 Make test categories composable
Summary:
Previously the add_test_categories would simply overwrite the current set of categories for a
method. This change makes the decorator truly "add" categories, by extending the current set of
categories instead of replacing it.

To do this, I have:
- replaced the getCategories() property on a method (which was itself a method), with a simple
  list property "categories". This makes add_test_categories easier to implement, and test
  categories isn't something which should change between calls anyway.
- rewritten the getCategoriesForTest function to merge method categories with the categories of
  the test case. Previously, it would just use the method categories if they were present. I have
  also greatly simplified this method. Originally, it would use a lot of introspection to enable
  it being called on various types of objects. Based on my tests, it was only ever being called
  on a test case. The new function uses much less introspection then the preivous one, so we
  should easily catch any stray uses, if there are any, as they will generate exceptions now.

Reviewers: zturner, tfiala, tberghammer

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15451

llvm-svn: 255493
2015-12-14 13:17:18 +00:00
Zachary Turner
742afdb3d2 Remove -S option from dotest.py.
llvm-svn: 255361
2015-12-11 19:21:49 +00:00
Todd Fiala
9187f27e32 Add test event marking a test as explicitly eligible for rerun if it is marked flakey.
This will be used in a future change to support rerunning flakey tests
that hit a test result isue in a low-load, single worker test runner phase.

This is implemented as an additive-style event rather than being
evaluated and added to the start_test event because the decorator code
only runs after the start_test event is created and sent.  i.e.
LLDBTestResult.startTest() runs before the test method decorators run.

llvm-svn: 255351
2015-12-11 18:06:47 +00:00
Kamil Rytarowski
f5d34b7b9d Add NetBSD support in the buildDriver and buildLibrary routines
Summary: NetBSD is like FreeBSD and Linux in these routines.

Reviewers: clay.chang, tfiala, emaste, joerg

Subscribers: lldb-commits, emaste

Differential Revision: http://reviews.llvm.org/D15374

llvm-svn: 255308
2015-12-10 22:56:56 +00:00
Zachary Turner
b08ab72427 Remove -w option from dotest.py.
llvm-svn: 255275
2015-12-10 18:50:32 +00:00
Zachary Turner
76972031fa Remove -k command line option from dotest.py.
This is part of an effort to remove unused command line options.

llvm-svn: 255143
2015-12-09 19:45:51 +00:00
Pavel Labath
7ead0b937c XFAIL TestReturnValue for remote Windows->Android tests
this also adds the ability to match the host platform to the expectedFailureAll decorator.

llvm-svn: 255105
2015-12-09 10:54:18 +00:00
Zachary Turner
d865c6b707 Remove the -c option from dotest.py.
This seems to be a legacy relic from days gone by where the
remote test suite runner operated completely differently than it
does today.  git blames and comments traced this functionality
back to about 2012, and nobody seems to know anything about it
now.

llvm-svn: 255060
2015-12-08 22:15:48 +00:00
Zachary Turner
2155d5d301 Remove the -b option from dotest.py
This removes the blacklist option as part of an effort to remove
unused / unmaintained command line options from the test suite.

llvm-svn: 255040
2015-12-08 18:43:16 +00:00
Zachary Turner
aad25fb9a5 Remove +b option from dotest.py
llvm-svn: 255037
2015-12-08 18:36:05 +00:00
Tamas Berghammer
ccd6cffba3 Modify "platform connect" to connect to processes as well
The standard remote debugging workflow with gdb is to start the
application on the remote host under gdbserver (e.g.: gdbserver :5039
a.out) and then connect to it with gdb.

The same workflow is supported by debugserver/lldb-gdbserver with a very
similar syntax but to access all features of lldb we need to be
connected also to an lldb-platform instance running on the target.

Before this change this had to be done manually with starting a separate
lldb-platform on the target machine and then connecting to it with lldb
before connecting to the process.

This change modifies the behavior of "platform connect" with
automatically connecting to the process instance if it was started by
the remote platform. With this command replacing gdbserver in a gdb
based worflow is usually as simple as replacing the command to execute
gdbserver with executing lldb-platform.

Differential revision: http://reviews.llvm.org/D14952

llvm-svn: 255016
2015-12-08 14:08:19 +00:00
Zachary Turner
606e3a5221 Get rid of global variables in dotest.py
This moves all the global variables into a separate module called
`configuration`.  This has a number of advantages:

1. Configuration data is centrally maintained so it's easy to get
   a high level overview of what configuration data the test suite
   makes use of.
2. The method of sharing configuration data among different parts
   of the test suite becomes standardized.  Previously we would
   put some things into the `lldb` module, some things into the
   `lldbtest_config` module, and some things would not get shared.
   Now everything is shared through one module and is available to
   the entire test suite.
3. It opens the door to moving some of the initialization code into
   the `configuration` module, simplifying the implementation of
   `dotest.py`.

There are a few stragglers that didn't get converted over to using
the `configuration` module in this patch, because it would have grown
the size of the patch unnecessarily.  This includes everything
currently in the `lldbtest_config` module, as well as the
`lldb.remote_platform` variable.  We can address these in the future.

llvm-svn: 254982
2015-12-08 01:15:30 +00:00
Kamil Rytarowski
49f9fb8d26 Add initial NetBSD support in lldbsuite/test/lldbtest.py
Summary:
Add new functions:

  - expectedFailureNetBSD()
  - expectedFlakeyNetBSD()
  - skipIfNetBSD()

Add new NetBSD entry in:

  - getPlatform()
  - getHostPlatform()

Assume that libc++ is installed and use the GNU toolchain

Reviewers: joerg, emaste, tfiala, clayborg

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D15262

llvm-svn: 254948
2015-12-07 21:25:57 +00:00
Kamil Rytarowski
0b655da7e4 Define new builder_netbsd
Summary: This is used in tests.

Reviewers: emaste, tfiala, clayborg

Subscribers: zturner, lldb-commits, joerg

Differential Revision: http://reviews.llvm.org/D15065

llvm-svn: 254853
2015-12-05 18:46:56 +00:00
Siva Chandra
7dcad3178b Revert "Make skipIf support the not_in function (second attempt)."
Summary: This reverts commit 70dca28976ee8137acce2cc203dd394f4d761276.

Reviewers: amccarth, zturner

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14881

llvm-svn: 253704
2015-11-20 20:30:36 +00:00
Adrian McCarthy
4d5d1dd6d5 Make skipIf support the not_in function (second attempt).
llvm-svn: 253683
2015-11-20 18:14:24 +00:00
Siva Chandra
d2e90129d9 Revert "Make skipIf support the not_in function."
Summary:
This reverts commit 251965377bdfb6227eea42c12a792c059e4e8a4b
as a test marked "skipIf(compiler='gcc')" runs when testing with GCC.

Reviewers: amccarth

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14852

llvm-svn: 253631
2015-11-20 01:54:24 +00:00
Adrian McCarthy
257da8ea50 Make skipIf support the not_in function.
llvm-svn: 253623
2015-11-20 00:11:38 +00:00
Enrico Granata
55d99f0e7c Cleanup work required to get the ASAN tests to run in the new test suite infrastructure; the tests are now xfailed on Darwin pending investigation
llvm-svn: 253604
2015-11-19 21:45:07 +00:00
Ying Chen
ca922bb9b9 Support unix-abstract-connect scheme as platform url in lldb testsuite
Reviewers: ovyalov

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14765

llvm-svn: 253488
2015-11-18 19:03:20 +00:00
Zachary Turner
48ef8d4c37 Fix some issues with swig & string conversion.
This patch fixes two issues:

1) Popen needs to be used with universal_newlines=True by default.
   This elicits automatic decoding from bytes -> string in Py3,
   and has no negative effects in other Py versions.
2) The swig typemaps for converting between string and (char*, int)
   did not work correctly when the length of the string was 0,
   indicating an error.  In this case we would try to construct a
   string from uninitialized data.
3) Ironically, the bug mentioned in #2 led to a test passing on
   Windows that was actually broken, because the test was written
   such that the assertion was never even getting checked, so it
   passed by default.  So we additionally fix this test to also
   fail if the method errors.  By fixing this test it's now broken
   on Windows, so we also xfail it.

llvm-svn: 253487
2015-11-18 18:40:16 +00:00
Zachary Turner
ba1057022c Python 3 - Skip a certain test for a particular (swig,python) combo.
Current versions of SWIG have a bug with Python 3 that causes
Python to assert when iterating over a generator.  This patch
skips the test for the right combination of Python version and
SWIG version.  I'm attempting to upstream a patch to SWIG to
fix this in a subsequent as-of-yet unreleased version, but
I don't know how long that will take.

llvm-svn: 253273
2015-11-16 23:58:20 +00:00
Ying Chen
0c35282c65 Re-Apply "Add a "not_in()" function you can apply to the list type arguments to expectedFailureAll ..." with fix
Summary:
- Re-Commit r253106
- Initialize self.debug_info in Base::setUp()
- Fix argument order when calling check_list_or_lambda for compiler

Reviewers: jingham

Subscribers: lldb-commits

Differential Revision: http://reviews.llvm.org/D14673

llvm-svn: 253272
2015-11-16 23:41:02 +00:00
Zachary Turner
abdb839228 Add the ability to xfail or skip based on swig / python version.
llvm-svn: 253263
2015-11-16 22:40:30 +00:00
Pavel Labath
311cc7d133 Revert "Add a "not_in()" function you can apply to the list type arguments to expectedFailureAll to reverse"
This reverts commit r253106.

llvm-svn: 253197
2015-11-16 11:11:10 +00:00
Jim Ingham
6d048942c5 Add a "not_in()" function you can apply to the list type arguments to expectedFailureAll to reverse
the sense of the test.

llvm-svn: 253106
2015-11-14 00:20:33 +00:00
Zachary Turner
8d13fab183 Python 3 - Don't use unbuffered I/O in text mode.
This is unsupported in Python 3.  This could also have been fixed
by using "wb" instead of "w", but it doesn't seem like writing the
session log absolutely *needs* to be unbuffered.

llvm-svn: 252381
2015-11-07 01:08:15 +00:00
Zachary Turner
5cb8e67b17 Don't use module internal implementation details in our decorators.
We tried implementing something akin to a conditionalExpectedFailure
decorator for unittest2.  We did this by making use of some
implementation details of the unittest2 module.  In an effort to make
this work with unittest, this patch removes the reliance on the
implementation details.  I have a hard time wrapping my head around
how this all works with the deeply nested decorators, but the spirit
of the patch here is to do do the following: If the condition function
is true, use the original unittest2.expectedFailure decorator.  Otherwise
don't use any decorator, just call the test function.

Differential Revision: http://reviews.llvm.org/D14406
Reviewed By: tberghammer, labath

llvm-svn: 252326
2015-11-06 18:14:42 +00:00
Zachary Turner
c1b7cd72db Python 3 - Turn on absolute imports, and fix existing imports.
Absolute imports were introduced in Python 2.5 as a feature
(e.g. from __future__ import absolute_import), and made default
in Python 3.

When absolute imports are enabled, the import system changes in
a couple of ways:

1) The `import foo` syntax will *only* search sys.path.  If `foo`
   isn't in sys.path, it won't be found.  Period.  Without absolute
   imports, the import system will also search the same directory
   that the importing file resides in, so that you can easily
   import from the same folder.

2) From inside a package, you can use a dot syntax to refer to higher
   levels of the current package.  For example, if you are in the
   package lldbsuite.test.utility, then ..foo refers to
   lldbsuite.test.foo.  You can use this notation with the
   `from X import Y` syntax to write intra-package references.  For
   example, using the previous locationa s a starting point, writing
   `from ..support import seven` would import lldbsuite.support.seven

Since this is now the default behavior in Python 3, this means that
importing from the same directory with `import foo` *no longer works*.
As a result, the only way to have portable code is to force absolute
imports for all versions of Python.

See PEP 0328 [https://www.python.org/dev/peps/pep-0328/] for more
information about absolute and relative imports.

Differential Revision: http://reviews.llvm.org/D14342
Reviewed By: Todd Fiala

llvm-svn: 252191
2015-11-05 19:22:28 +00:00
Enrico Granata
5f92a130ff Teach LLDB how to directly launch processes on the iOS simulator
This allows for command-line debugging of iOS simulator binaries (as long as UI is not required, or a full UI simulator has previously been otherwise launched), as well as execution of the LLDB test suite on the iOS simulator

This is known to compile on OSX 10.11 GM - feedback from people on other platforms and/or older versions of OSX as to the buildability of this code is greatly appreciated

llvm-svn: 252112
2015-11-05 00:46:25 +00:00
Zachary Turner
19474e1801 Remove use_lldb_suite from the package, and don't import it anymore.
This module was originally intended to be imported by top-level
scripts to be able to find the LLDB packages and third party
libraries.  Packages themselves shouldn't need to import it,
because by the time it gets into the package, the top-level
script should have already done this.  Indeed, it was just
adding the same values to sys.path multiple times, so this
patch is essentially no functional change.

To make sure it doesn't get re-introduced, we also delete the
`use_lldb_suite` module from `lldbsuite/test`, although the
original copy still remains in `lldb/test`

llvm-svn: 251963
2015-11-03 19:20:39 +00:00
Zachary Turner
95c453a221 Tighten up sys.path, and use absolute imports everywhere.
For convenience, we had added the folder that dotest.py was in
to sys.path, so that we could easily write things like
`import lldbutil` from anywhere and any test.  This introduces
a subtle problem when using Python's package system, because when
unittest2 imports a particular test suite, the test suite is detached
from the package.  Thus, writing "import lldbutil" from dotest imports
it as part of the package, and writing the same line from a test
does a fresh import since the importing module was not part of
the same package.

The real way to fix this is to use absolute imports everywhere.  Instead
of writing "import lldbutil", we need to write "import
lldbsuite.test.util".  This patch fixes up that and all other similar
cases, and additionally removes the script directory from sys.path
to ensure that this can't happen again.

llvm-svn: 251886
2015-11-03 02:06:18 +00:00
Zachary Turner
c432c8f856 Move lldb/test to lldb/packages/Python/lldbsuite/test.
This is the conclusion of an effort to get LLDB's Python code
structured into a bona-fide Python package.  This has a number
of benefits, but most notably the ability to more easily share
Python code between different but related pieces of LLDB's Python
infrastructure (for example, `scripts` can now share code with
`test`).

llvm-svn: 251532
2015-10-28 17:43:26 +00:00