This is triggered when we are compiling an implementation of a module,
it has relative includes to a VFS-mapped module with umbrella headers.
Currently we will find the real path to headers under the umbrella directory,
but the umbrella directories are using virtual path.
rdar://27951255
Thanks Ben and Richard for reviewing the patch!
Differential Revision: http://reviews.llvm.org/D23858
llvm-svn: 279838
getModuleContainingLocation ends up on the hot-path for typical C code
which can lead to calls to getFileIDSlow.
To speed this up, short circuit inferModuleFromLocation when there
aren't any modules, implicit or otherwise.
This shaves 4-5% build time when building the linux kernel.
llvm-svn: 269687
Clang performs directory walk while searching headers inside modules by
using the ::sys::fs instead of ::vfs. This prevents any code that uses
the VFS (e.g, reproducer scripts) to actually find such headers, since
the VFS will never be searched for those.
Change these places to use vfs::recursive_directory_iterator and
vfs::directory_iterator instead.
Differential Revision: http://reviews.llvm.org/D20266
rdar://problem/25880368
llvm-svn: 269661
(1) Collect headers under inner frameworks (frameworks inside other
other frameworks).
(2) Make sure we also collect the right header files inside them.
More info on (2):
Consider a dummy framework module B, with header Frameworks/B/B.h. Now
consider that another framework A, with header Frameworks/A/A.h, has a
layout with a inner framework Frameworks/A/Frameworks/B/B.h, where the
"B/B.h" part is a symlink for Frameworks/B/B.h. Also assume that
Frameworks/A/A.h includes <B/B.h>.
When parsing header Frameworks/A/A.h, framework module lookup is
performed in search for B, and it happens that
"Frameworks/A/Frameworks/B/B.h" path is registered in the module instead
of real "Frameworks/B/B.h". This occurs because
"Frameworks/A/Frameworks/B/B.h" is scanned first by the FileManager,
when looking for inner framework modules under Frameworks/A/Frameworks.
This makes Frameworks/A/Frameworks/B/B.h the default cached named inside
the FileManager for the B.h file UID.
This leads to modules being built without consistent paths to underlying
header files. This is usually not a problem in regular compilation flow,
but it's an issue when running the crash reproducer. The issue is that
clangs collect "Frameworks/A/Frameworks/B/B.h" but not
"Frameworks/B/B.h" into the VFS, leading to err_mmap_umbrella_clash. So
make sure we also collect the original header.
Differential Revision: http://reviews.llvm.org/D20194
rdar://problem/25880368
llvm-svn: 269502
Use a StringRef instead of a FileEntry in the moduleMapAddHeader
callback to allow more flexibility on what to collect on further
patches. This changes the interface I introduced in r264971.
llvm-svn: 268819
The current ModuleDependencyCollector has a AST listener to collect
header files present in loaded modules, but this isn't enough to collect
all headers needed in the crash reproducer. One of the reasons is that
the AST writer doesn't write symbolic link header paths in the pcm modules,
this makes the listeners on the reader only able to collect the real files.
Since the module maps could contain submodules that use headers which
are symbolic links, not collecting those forbid the reproducer scripts
to regen the modules.
For instance:
usr/include/module.map:
...
module pthread {
header "pthread.h"
export *
module impl {
header "pthread_impl.h"
export *
}
}
...
usr/include/pthread/pthread_impl.h
usr/include/pthread_impl.h -> pthread/pthread_impl.h
The AST dump for the module above:
<SUBMODULE_HEADER abbrevid=6/> blob data = 'pthread_impl.h'
<SUBMODULE_TOPHEADER abbrevid=7/> blob data = '/<path_to_sdk>/usr/include/pthread/pthread_impl.h'
Note that we don't have "usr/include/pthread_impl.h" which is requested
by the module.map in case we want to reconstruct the module in the
reproducer. The reason the original symbolic link path isn't used is
because the headers are kept by name and requested through the
FileManager, which unique files and returns the real path only.
To fix that, add a callback to be invoked everytime a header is added
while parsing module maps and hook that up to the module dependecy
collector. This callback is only registered when generating the
reproducer.
Differential Revision: http://reviews.llvm.org/D18585
rdar://problem/24499339
llvm-svn: 264971
Since it's provided by the compiler. This allows a system module map
file to declare a module for it.
No test change for cstd.m, since stdatomic.h doesn't function without a
relatively complete stdint.h and stddef.h, which tests using this module
don't provide.
rdar://problem/24931246
llvm-svn: 263076
option. Previously these options could both be used to specify that you were
compiling the implementation file of a module, with a different set of minor
bugs in each case.
This change removes -fmodule-implementation-of, and instead tracks a flag to
determine whether we're currently building a module. -fmodule-name now behaves
the same way that -fmodule-implementation-of previously did.
llvm-svn: 261372
When linking against text-based dynamic library SDKs the library name of a
framework has now more than one possible filename extensions. This fix tests for
both possible extensions (none, and .tbd).
This fixes rdar://problem/20609975
llvm-svn: 253060
Summary:
If a module was unavailable (either a missing requirement on the module
being imported, or a missing file anywhere in the top-level module (and
not dominated by an unsatisfied `requires`)), we would silently treat
inclusions as textual. This would cause all manner of crazy and
confusing errors (and would also silently "work" sometimes, making the
problem difficult to track down).
I'm really not a fan of the `M->isAvailable(getLangOpts(), getTargetInfo(),
Requirement, MissingHeader)` function; it seems to do too many things at
once, but for now I've done things in a sort of awkward way.
The changes to test/Modules/Inputs/declare-use/module.map
were necessitated because the thing that was meant to be tested there
(introduced in r197805) was predicated on silently falling back to textual
inclusion, which we no longer do.
The changes to test/Modules/Inputs/macro-reexport/module.modulemap
are just an overlooked missing header that seems to have been missing since
this code was committed (r213922), which is now caught.
Reviewers: rsmith, benlangmuir, djasper
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D10423
llvm-svn: 245228
This preserves backwards compatibility for two hacks in the Darwin
system module map files:
1. The use of 'requires excluded' to make headers non-modular, which
should really be mapped to 'textual' now that we have this feature.
2. Silently removes a bogus cplusplus requirement from IOKit.avc.
Once we start diagnosing missing requirements and headers on
auto-imports these would have broken compatibility with existing Darwin
SDKs.
llvm-svn: 244912
We use findModuleForHeader() in several places, but in header search we
were not calling it when a framework module didn't show up with the
expected name, which would then lead to unexpected non-modular includes.
Now we will find the module unconditionally for frameworks. For regular
frameworks, we use the spelling of the module name from the module map
file, and for inferred ones we use the canonical directory name.
In the future we might want to lock down framework modules sufficiently
that these name mismatches cannot happen.
rdar://problem/20465870
llvm-svn: 241258
The patch is generated using this command:
$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
work/llvm/tools/clang
To reduce churn, not touching namespaces spanning less than 10 lines.
llvm-svn: 240270
We used to have a flag to enable module maps, and two more flags to enable
implicit module maps. This is all redundant; we don't need any flag for
enabling module maps in the abstract, and we don't usually have -fno- flags for
-cc1. We now have just a single flag, -fimplicit-module-maps, that enables
implicitly searching the file system for module map files and loading them.
The driver interface is unchanged for now. We should probably rename
-fmodule-maps to -fimplicit-module-maps at some point.
llvm-svn: 239789
The RequestingModule argument was unused and always its default value of
nullptr.
Also move a declaration closer to its use, and range-for'ify.
llvm-svn: 239453
With this change, enabling -fmodules-local-submodule-visibility results in name
visibility rules being applied to submodules of the current module in addition
to imported modules (that is, names no longer "leak" between submodules of the
same top-level module). This also makes it much safer to textually include a
non-modular library into a module: each submodule that textually includes that
library will get its own "copy" of that library, and so the library becomes
visible no matter which including submodule you import.
llvm-svn: 237473
It has no place there; it's not a property of the Module, and it makes
restoring the visibility set when we leave a submodule more difficult.
llvm-svn: 236300
Now that SmallString is a first-class citizen, most SmallString::str()
calls are not required. This patch removes a whole bunch of them, yet
there are lots more.
There are two use cases where str() is really needed:
1) To use one of StringRef member functions which is not available in
SmallString.
2) To convert to std::string, as StringRef implicitly converts while
SmallString do not. We may wish to change this, but it may introduce
ambiguity.
llvm-svn: 232622
check that private headers are in a list matching the role. (We can't perform
the opposite checks for non-private headers because we infer those.)
llvm-svn: 231728
If a module map contains
framework module * [extern_c] {}
We will now infer [extern_c] on the inferred framework modules (we
already inferred [system] as a special case).
llvm-svn: 225803
Original commit message:
[modules] Add experimental -fmodule-map-file-home-is-cwd flag to -cc1.
For files named by -fmodule-map-file=, and files found by 'extern module'
directives, this flag specifies that we should resolve filenames relative to
the current working directory rather than relative to the directory in which
the module map file resides. This is aimed at fixing path handling, in
particular for relative -I paths, when building modules that represent
components of the current project (rather than libraries installed on the
current system, which the current project has as dependencies, where we'd
typically expect the module map files to be looked up implicitly).
llvm-svn: 223913
For files named by -fmodule-map-file=, and files found by 'extern module'
directives, this flag specifies that we should resolve filenames relative to
the current working directory rather than relative to the directory in which
the module map file resides. This is aimed at fixing path handling, in
particular for relative -I paths, when building modules that represent
components of the current project (rather than libraries installed on the
current system, which the current project has as dependencies, where we'd
typically expect the module map files to be looked up implicitly).
llvm-svn: 223753
rather than trying to extract this information from the FileEntry after the
fact.
This has a number of beneficial effects. For instance, diagnostic messages for
failed module builds give a path relative to the "module root" rather than an
absolute file path, and the contents of the module includes file is no longer
dependent on what files the including TU happened to inspect prior to
triggering the module build.
llvm-svn: 223095
This was not a real header role, and was never exposed to clients of ModuleMap.
Remove the enumeration value for it and track it as marking the header as
'known' rather than creating an extra KnownHeader entry that *every single*
client ignores.
llvm-svn: 220460
This allows a module to specify that it logically contains a file, but that
said file is non-modular and intended for textual inclusion. This allows
layering checks to work properly in the presence of such files.
llvm-svn: 220448
#include_next interacts poorly with modules: it depends on where in the list of
include paths the current file was found. Files covered by module maps are not
found in include search paths when building the module (and are not found in
include search paths when @importing the module either), so this isn't really
meaningful. Instead, we fake up the result that #include_next *should* have
given: find the first path that would have resulted in the given file being
picked, and search from there onwards.
llvm-svn: 220177
We already verified the primary module map file (either the one that
defines the top-level module, or the one that allows inferring it if it
is an inferred framework module). Now we also verify any other module
map files that define submodules, such as when there is a
module.private.modulemap file.
llvm-svn: 215455
class Module. It's almost always going to be the same as
getContainingModule() for top-level modules, so just add a map to cover
the remaining cases. This lets us do less bookkeeping to keep the
ModuleMap fields up to date.
llvm-svn: 215268
Just because we can open a directory named "COcoa.framework" doesn't
mean we should provide a "COcoa" module on a case-insensitive filesystem.
llvm-svn: 212975
But keep -Wnon-modular-include-in-[framework-]module
This warning is too noisy and doesn't really indicate a problem for most
people. Even though it would only really affect people using
-Weverything, that seems bad so remove it.
llvm-svn: 208345
Warn on non-modular includes in various contexts.
-Wnon-modular-include
-Wnon-modular-include-in-module
-Wnon-modular-include-in-framework-module
Where each group is a subgroup of those above it.
llvm-svn: 208004
Otherwise including a header in your source file that is not included by
framework's umbrella header will silently add an empty submodule with that
name.
is automatically translated to
@import Foo.NotInModule;
which then would have succeeded because the inferred module map
contained an empty submodule called NotInModule.
llvm-svn: 207024
Unless they are in submodules that aren't available anyway, due to
requirements not being met. Also, mark children as unavailable when the
parent is.
llvm-svn: 206664
To differentiate between two modules with the same name, we will
consider the path the module map file that they are defined by* part of
the ‘key’ for looking up the precompiled module (pcm file).
Specifically, this patch renames the precompiled module (pcm) files from
cache-path/<module hash>/Foo.pcm
to
cache-path/<module hash>/Foo-<hash of module map path>.pcm
In addition, I’ve taught the ASTReader to re-resolve the names of
imported modules during module loading so that if the header search
context changes between when a module was originally built and when it
is loaded we can rebuild it if necessary. For example, if module A
imports module B
first time:
clang -I /path/to/A -I /path/to/B ...
second time:
clang -I /path/to/A -I /different/path/to/B ...
will now rebuild A as expected.
* in the case of inferred modules, we use the module map file that
allowed the inference, not the __inferred_module.map file, since the
inferred file path is the same for every inferred module.
llvm-svn: 206201
This name, while more verbose, plays more nicely with tools that use
file extensions to determine file types. The existing spelling
'module.map' will continue to work, but the new spelling will take
precedence.
In frameworks, this new filename will only go in a new 'Modules'
sub-directory.
Similarly, add a module.private.modulemap corresponding to
module_private.map.
llvm-svn: 204261
that implicitly converts to 'bool' (such as pointers, and the first operand of
?:). Clean up issues found by this. Patch by Stephan Tolksdorf!
llvm-svn: 203735
Add module dependencies to the dependency files created by -MD/-MMD/etc.
by attaching an ASTReaderListener that will call into the dependency
file generator when a module input file is seen in the serialized AST.
llvm-svn: 203208
If a header file belonging to a certain module is not found on the
filesystem, that header gets marked as unavailable. Now, the layering
warning (-fmodules-decluse) should still warn about headers of this
module being wrongfully included. Currently, headers belonging to those
modules are just treated as not belonging to modules at all which means
they can be included freely from everywhere.
To implement this (somewhat) cleanly, I have moved most of the layering
checks into the ModuleMap. This will also help with showing FixIts
later.
llvm-svn: 197805
Instead, mark the module as unavailable so that clang errors as soon as
someone tries to build this module.
This works towards the long-term goal of not stat'ing the header files at all
while reading the module map and instead read them only when the module is
being built (there is a corresponding FIXME in parseHeaderDecl()). However, it
seems non-trivial to get there and this unblock us and moves us into the right
direction.
Also changed the implementation to reuse the same DiagnosticsEngine.
llvm-svn: 197485
Instead, mark the module as unavailable so that clang errors as soon as
someone tries to build this module.
A better long-term strategy might be to not stat the header files at all
while reading the module map and instead read them only when the module
is being built (there is a corresponding FIXME in parseHeaderDecl()).
However, it seems non-trivial to get there and this would be a temporary
solution to unblock us.
Also changed the implementation to reuse the same DiagnosticsEngine as
otherwise warnings can't be enabled or disabled with command-line flags.
llvm-svn: 197388
Includes might always pull in arbitrary header or data files outside of
modules. Among others, this includes builtin includes, which do not have
a module (story) yet.
Also cleanup implementation of ModuleMap::findModuleForHeader() to be
non-recursive.
llvm-svn: 197034
In order to make the migration to modules easier, it seems to be helpful
to allow a 1:1 mapping between target names of a current build system
and the corresponding C++ modules. As such targets commonly contain
characters like "-". ":" and "/", allowing arbitrary quote-escaped
strings seems to be a straightforward option.
After several offline discussions, the precise mechanisms for C++
module names especially regarding submodules and import statements has
yet to be determined. Thus, this patch only enables string literals as
names inside the module map files which can be used by automatic module
import (through #include).
Also improve the error message on missing use-declarations.
llvm-svn: 196573
requires ! feature
The purpose of this is to allow (for instance) the module map for /usr/include
to exclude <tgmath.h> and <complex.h> when building in C++ (these headers are
instead provided by the C++ standard library in this case, and the glibc C
<tgmath.h> header would otherwise try to include <complex.h>, resulting in a
module cycle).
llvm-svn: 193549
This allows using virtual file mappings on the original SourceManager to
map in virtual module.map files. Without this patch, the ModuleMap
search will find a module.map file (as the FileEntry exists in the
FileManager), but will be unable to get the content from the
SourceManager (as ModuleMap previously created its own SourceManager).
Two problems needed to be fixed which this patch exposed:
1. Storing the inferred module map
When writing out a module, the ASTWriter stores the names of the files
in the main source manager; when loading the AST again, the ASTReader
errs out if such a file is found missing, unless it is overridden.
Previously CompilerInstance's compileModule method would store the
inferred module map to a temporary file; the problem with this approach
is that now that the module map is handled by the main source manager,
the ASTWriter stores the name of the temporary module map as source to
the compilation; later, when the module is loaded, the temporary file
has already been deleted, which leads to a compilation error. This patch
changes the inferred module map to instead inject a virtual file into
the source manager. This both saves some disk IO, and works with how the
ASTWriter/ASTReader handle overridden source files.
2. Changing test input in test/Modules/Inputs/*
Now that the module map file is handled by the main source manager, the
VerifyDiagnosticConsumer will not ignore diagnostics created while
parsing the module map file. The module test test/Modules/renamed.m uses
-I test/Modules/Inputs and triggers recursive loading of all module maps
in test/Modules/Inputs, some of which had conflicting names, thus
leading errors while parsing the module maps. Those diagnostics already
occur on trunk, but before this patch they would not break the test, as
they were ignored by the VerifyDiagnosticConsumer. This patch thus
changes the module maps that have been recently introduced which broke
the invariant of compatible modules maps in test/Modules/Inputs.
llvm-svn: 193314
This patch changes two things:
a) Allow a header to be part of multiple modules. The reasoning is that
in existing codebases that have a module-like build system, the same
headers might be used in several build targets. Simple reasons might be
that they defined different classes that are declared in the same
header. Supporting a header as a part of multiple modules will make the
transistion easier for those cases. A later step in clang can then
determine whether the two modules are actually compatible and can be
merged and error out appropriately. The later check is similar to what
needs to be done for template specializations anyway.
b) Allow modules to be stored in a directory tree separate from the
headers they describe.
Review: http://llvm-reviews.chandlerc.com/D1951
llvm-svn: 193151
Review: http://llvm-reviews.chandlerc.com/D1546.
I have picked up this patch form Lawrence
(http://llvm-reviews.chandlerc.com/D1063) and did a few changes.
From the original change description (updated as appropriate):
This patch adds a check that ensures that modules only use modules they
have so declared. To this end, it adds a statement on intended module
use to the module.map grammar:
use module-id
A module can then only use headers from other modules if it 'uses' them.
This enforcement is off by default, but may be turned on with the new
option -fmodules-decluse.
When enforcing the module semantics, we also need to consider a source
file part of a module. This is achieved with a compiler option
-fmodule-name=<module-id>.
The compiler at present only applies restrictions to the module directly
being built.
llvm-svn: 191283
This patch is the first step to make module-map-files modular (instead
of requiring a single "module.map"-file per include directory). This
step adds a new "extern module" declaration that enables
module-map-files to reference one another along with a very basic
implementation.
The next steps are:
* Combine this with the use-declaration (from
http://llvm-reviews.chandlerc.com/D1546) in order to only load module
map files required for a specific compilation.
* Add an additional flag to start with a specific module-map-file (instead
of requiring there to be at least one "module.map").
Review: http://llvm-reviews.chandlerc.com/D1637
llvm-svn: 190497
