Nadav Rotem
26bf9a0c75
SLPVectorizer: Reduce the compile time of the consecutive store lookup.
...
Process groups of stores in chunks of 16.
llvm-svn: 186420
2013-07-16 15:25:17 +00:00
Nadav Rotem
1c1d6c1666
PR16628: Fix a bug in the code that merges compares.
...
Compares return i1 but they compare different types.
llvm-svn: 186359
2013-07-15 22:52:48 +00:00
Nadav Rotem
d9f3f4548e
SLPVectorizer: change the order in which we search for vectorization candidates. Do stores first and PHIs second.
...
llvm-svn: 186277
2013-07-14 06:15:46 +00:00
Nadav Rotem
89c41bf06a
SLPVectorizer: Sink and enable CSE for ExtractElements.
...
llvm-svn: 186145
2013-07-12 06:09:24 +00:00
Nadav Rotem
fa3c2db211
SLPVectorize: Replace the code that checks for vectorization candidates in successor blocks with code that scans PHINodes.
...
Before we could vectorize PHINodes scanning successors was a good way of finding candidates. Now we can vectorize the phinodes which is simpler.
llvm-svn: 186139
2013-07-12 00:04:18 +00:00
Nadav Rotem
db06b139fd
Remove an argument that we dont use anymore.
...
llvm-svn: 186116
2013-07-11 20:56:13 +00:00
Nadav Rotem
08efb262a9
Fix a warning.
...
llvm-svn: 186064
2013-07-11 05:39:02 +00:00
Nadav Rotem
b8dd66f655
SLPVectorizer: refactor the code that places extracts. Place the code that decides where to put extracts in the build-tree phase. This allows us to take the cost of the extracts into account.
...
llvm-svn: 186058
2013-07-11 04:54:05 +00:00
Nadav Rotem
d7b574e5b3
Fix PR16571, which is a bug in the code that checks that all of the types in the bundle are uniform.
...
llvm-svn: 185970
2013-07-09 21:38:08 +00:00
Nadav Rotem
861bef7dd0
Set the default insert point to the first instruction, and not to end()
...
llvm-svn: 185953
2013-07-09 17:55:36 +00:00
Nadav Rotem
c9c57518ab
This patch changes the saved IRBuilder insert point from BasicBlock::iterator to AssertingVH.
...
Commit 185883 fixes a bug in the IRBuilder that should fix the ASan bot. AssertingVH can help in exposing some RAUW problems.
Thanks Ben and Alexey!
llvm-svn: 185886
2013-07-08 23:31:13 +00:00
Nadav Rotem
2ee35771a8
Clear the builder insert point between tree-vectorization phases.
...
llvm-svn: 185777
2013-07-07 14:57:18 +00:00
Nadav Rotem
2041b742d4
SLPVectorizer: Implement DCE as part of vectorization.
...
This is a complete re-write if the bottom-up vectorization class.
Before this commit we scanned the instruction tree 3 times. First in search of merge points for the trees. Second, for estimating the cost. And finally for vectorization.
There was a lot of code duplication and adding the DCE exposed bugs. The new design is simpler and DCE was a part of the design.
In this implementation we build the tree once. After that we estimate the cost by scanning the different entries in the constructed tree (in any order). The vectorization phase also works on the built tree.
llvm-svn: 185774
2013-07-07 06:57:07 +00:00
Craig Topper
af0dea1347
Use SmallVectorImpl::iterator/const_iterator instead of SmallVector to avoid specifying the vector size.
...
llvm-svn: 185606
2013-07-04 01:31:24 +00:00
Nadav Rotem
0a25727f31
We preserve the CFG and some of the analysis passes.
...
llvm-svn: 185251
2013-06-29 05:38:15 +00:00
Nadav Rotem
e00343446c
Update docs.
...
llvm-svn: 185250
2013-06-29 05:37:19 +00:00
Nadav Rotem
060be733a5
SLP Vectorizer: Add support for trees with external users.
...
To support this we have to insert 'extractelement' instructions to pick the right lane.
We had this functionality before but I removed it when we moved to the multi-block design because it was too complicated.
llvm-svn: 185230
2013-06-28 22:07:09 +00:00
Nadav Rotem
8edefb3665
No need to use a Set when a vector would do.
...
llvm-svn: 185047
2013-06-27 00:14:13 +00:00
Nadav Rotem
93f880fb77
SLP: When searching for vectorization opportunities scan the blocks in post-order because we grow chains upwards.
...
llvm-svn: 185041
2013-06-26 23:44:45 +00:00
Nadav Rotem
7f0d6d7975
SLP: Dont erase instructions during vectorization because it prevents the outerloops from iterating over the instructions.
...
llvm-svn: 185040
2013-06-26 23:43:23 +00:00
Nadav Rotem
4c5b2d1de6
Erase all of the instructions that we RAUWed
...
llvm-svn: 184969
2013-06-26 17:16:09 +00:00
Nadav Rotem
f4ca3994b8
Do not add cse-ed instructions into the visited map because we dont want to consider them as a candidate for replacement of instructions to be visited.
...
llvm-svn: 184966
2013-06-26 16:54:53 +00:00
Nadav Rotem
0794acc1da
SLPVectorizer: support slp-vectorization of PHINodes between basic blocks
...
llvm-svn: 184888
2013-06-25 23:04:09 +00:00
Nadav Rotem
3de032a3b6
Fix a typo in the code that collected the costs recursively.
...
llvm-svn: 184827
2013-06-25 05:30:56 +00:00
Nadav Rotem
9c7c997a7e
Rename the variable to fix a warning. Thanks Andy Gibbs.
...
llvm-svn: 184749
2013-06-24 15:59:47 +00:00
Nadav Rotem
210e86d7c4
SLP Vectorizer: Add support for vectorizing parts of the tree.
...
Untill now we detected the vectorizable tree and evaluated the cost of the
entire tree. With this patch we can decide to trim-out branches of the tree
that are not profitable to vectorizer.
Also, increase the max depth from 6 to 12. In the worse possible case where all
of the code is made of diamond-shaped graph this can bring the cost to 2**10,
but diamonds are not very common.
llvm-svn: 184681
2013-06-24 02:52:43 +00:00
Nadav Rotem
0323925d51
SLP Vectorizer: Fix a bug in the code that does CSE on the generated gather sequences.
...
Make sure that we don't replace and RAUW two sequences if one does not dominate the other.
llvm-svn: 184674
2013-06-23 21:57:27 +00:00
Nadav Rotem
78428401e9
SLP Vectorizer: Erase instructions outside the vectorizeTree method.
...
The RAII builder location guard is saving a reference to instructions, so we can't erase instructions during vectorization.
llvm-svn: 184671
2013-06-23 19:38:56 +00:00
Nadav Rotem
eb65e67eea
SLP Vectorizer: Implement a simple CSE optimization for the gather sequences.
...
llvm-svn: 184660
2013-06-23 06:15:46 +00:00
Nadav Rotem
80de0a28f1
SLP Vectorizer: Implement multi-block slp-vectorization.
...
Rewrote the SLP-vectorization as a whole-function vectorization pass. It is now able to vectorize chains across multiple basic blocks.
It still does not vectorize PHIs, but this should be easy to do now that we scan the entire function.
I removed the support for extracting values from trees.
We are now able to vectorize more programs, but there are some serious regressions in many workloads (such as flops-6 and mandel-2).
llvm-svn: 184647
2013-06-22 21:34:10 +00:00
Nadav Rotem
b488beefeb
Clang-format the SLP vectorizer. No functionality change.
...
llvm-svn: 184446
2013-06-20 17:54:36 +00:00
Nadav Rotem
14a89c5428
SLPVectorization: Add a basic support for cross-basic block slp vectorization.
...
We collect gather sequences when we vectorize basic blocks. Gather sequences are excellent
hints for vectorization of other basic blocks.
llvm-svn: 184444
2013-06-20 17:41:45 +00:00
Nadav Rotem
c41028a013
Change the debug type to match the debug type that is used by vecutils.cpp.
...
This change makes it easier to filter debug messages.
llvm-svn: 184440
2013-06-20 16:38:05 +00:00
Nadav Rotem
1f96427da0
Scan the successor blocks and use the PHI nodes as a hint for possible chain roots.
...
llvm-svn: 184201
2013-06-18 15:58:05 +00:00
Nadav Rotem
9e00eb38a2
SLPVectorizer: Change the order in which new instructions are added to the function.
...
We are not working on a DAG and I ran into a number of problems when I enabled the vectorizations of 'diamond-trees' (trees that share leafs).
* Imroved the numbering API.
* Changed the placement of new instructions to the last root.
* Fixed a bug with external tree users with non-zero lane.
* Fixed a bug in the placement of in-tree users.
llvm-svn: 182508
2013-05-22 19:47:32 +00:00
Nadav Rotem
0686e5cb05
Add a debug print
...
llvm-svn: 181647
2013-05-10 22:56:18 +00:00
Nadav Rotem
9feda6071a
Fix a typo
...
llvm-svn: 180806
2013-04-30 21:04:51 +00:00
Nadav Rotem
8aca44a623
Fix PR15800. Do not try to vectorize vectors and structs.
...
llvm-svn: 179960
2013-04-20 22:29:43 +00:00
Benjamin Kramer
4600bcc337
SLPVectorizer: Strength reduce SmallVectors to ArrayRefs.
...
Avoids a couple of copies and allows more flexibility in the clients.
llvm-svn: 179935
2013-04-20 09:49:10 +00:00
Nadav Rotem
ce2660d639
SLPVectorizer: Reduce the compile time by eliminating the search for some of the more expensive patterns. After this change will only check basic arithmetic trees that start at cmpinstr.
...
llvm-svn: 179933
2013-04-20 07:29:34 +00:00
Nadav Rotem
998e035cae
refactor tryToVectorizePair to a new method that supports vectorization of lists.
...
llvm-svn: 179932
2013-04-20 07:22:58 +00:00
Nadav Rotem
890387289e
Fix an unused variable warning.
...
llvm-svn: 179931
2013-04-20 06:40:28 +00:00
Nadav Rotem
83c7c41bc2
SLPVectorizer: Improve the cost model for loop invariant broadcast values.
...
llvm-svn: 179930
2013-04-20 06:13:47 +00:00
Nadav Rotem
dfe1c93ca4
Report the number of stores that were found in the debug message.
...
llvm-svn: 179929
2013-04-20 05:23:11 +00:00
Nadav Rotem
b9116e6966
SLPVectorizer: Make it a function pass and add code for hoisting the vector-gather sequence out of loops.
...
llvm-svn: 179562
2013-04-15 22:00:26 +00:00
Nadav Rotem
5d393c416f
SLPVectorizer: Add support for vectorizing trees that start at compare instructions.
...
llvm-svn: 179504
2013-04-15 04:25:27 +00:00
Nadav Rotem
54b413d157
SLPVectorizer: Add support for trees that don't start at binary operators, and add the cost of extracting values from the roots of the tree.
...
llvm-svn: 179475
2013-04-14 05:15:53 +00:00
Nadav Rotem
0b9cf8567b
SLPVectorizer: add initial support for reduction variable vectorization.
...
llvm-svn: 179470
2013-04-14 03:22:20 +00:00
Nadav Rotem
4da0ab1d68
Add debug prints.
...
llvm-svn: 179412
2013-04-12 21:11:14 +00:00
Nadav Rotem
73dffa4184
Make the SLP store-merger less paranoid about function calls. We check for function calls when we check if it is safe to sink instructions.
...
llvm-svn: 179207
2013-04-10 19:41:36 +00:00