Remake
Loading...
Searching...
No Matches
remake.cpp
Go to the documentation of this file.
1/* -*- mode: C++; indent-tabs-mode: t; c-basic-offset: 8; -*- */
2/**
3@mainpage Remake, a build system that bridges the gap between make and redo.
4
5As with <b>make</b>, <b>remake</b> uses a centralized rule file, which is
6named <b>Remakefile</b>. It contains rules with a <em>make</em>-like
7syntax:
8
9@verbatim
10target1 target2 ... : prerequisite1 prerequisite2 ...
11 shell script
12 that builds
13 the targets
14@endverbatim
15
16A target is known to be up-to-date if all its prerequisites are. If it
17has no known prerequisites yet the file already exits, it is assumed to
18be up-to-date. Obsolete targets are rebuilt thanks to the shell script
19provided by the rule.
20
21As with <b>redo</b>, <b>remake</b> supports dynamic dependencies in
22addition to these static dependencies. Whenever a script executes
23`remake prerequisite4 prerequisite5 ...`, these prerequisites are
24rebuilt if they are obsolete. (So <b>remake</b> acts like
25<b>redo-ifchange</b>.) Moreover, all the dependencies are stored in file
26<b>.remake</b> so that they are remembered in subsequent runs. Note that
27dynamic dependencies from previous runs are only used to decide whether a
28target is obsolete; they are not automatically rebuilt when they are
29obsolete yet a target depends on them. They will only be rebuilt once the
30dynamic call to <b>remake</b> is executed.
31
32In other words, the following two rules have almost the same behavior.
33
34@verbatim
35target1 target2 ... : prerequisite1 prerequisite2 ...
36 shell script
37
38target1 target2 ... :
39 remake prerequisite1 prerequisite2 ...
40 shell script
41@endverbatim
42
43(There is a difference if the targets already exist, have never been
44built before, and the prerequisites are either younger or obsolete, since
45the targets will not be rebuilt in the second case.)
46
47The above usage of dynamic dependencies is hardly useful. Their strength
48lies in the fact that they can be computed on the fly:
49
50@verbatim
51%.o : %.c
52 gcc -MMD -MF $@.d -o $@ -c $<
53 remake -r < $@.d
54 rm $@.d
55
56%.cmo : %.ml
57 ocamldep $< | remake -r $@
58 ocamlc -c $<
59
60after.xml: before.xml rules.xsl
61 xsltproc --load-trace -o after.xml rules.xsl before.xml 2> deps
62 remake `sed -n -e "\\,//,! s,^.*URL=\"\\‍([^\"]*\\‍).*\$,\\1,p" deps`
63 rm deps
64@endverbatim
65
66Note that the first rule fails if any of the header files included by
67a C source file has to be automatically generated. In that case, one
68should perform a first call to <b>remake</b> them before calling the
69compiler. (Dependencies from several calls to <b>remake</b> are
70cumulative, so they will all be remembered the next time.)
71
72\section sec-usage Usage
73
74Usage: <tt>remake <i>options</i> <i>targets</i></tt>
75
76Options:
77
78- `-B`, `--always-make`: Unconditionally make all targets.
79- `-d`: Echo script commands.
80- `-f FILE`: Read `FILE` as <b>Remakefile</b>.
81- `-j[N]`, `--jobs=[N]`: Allow `N` jobs at once;
82 infinite jobs with no argument.
83- `-k`, `--keep-going`: Keep going when some targets cannot be made.
84- `-r`: Look up targets from the dependencies on standard input.
85- `-s`, `--silent`, `--quiet`: Do not echo targets.
86
87\section sec-syntax Syntax
88
89Lines starting with a space character or a tabulation are assumed to be rule
90scripts. They are only allowed after a rule header.
91
92Lines starting with `#` are considered to be comments and are ignored.
93They do interrupt rule scripts though.
94
95Any other line is either a variable definition or a rule header. If such a
96line ends with a backslash, the following line break is ignored and the line
97extends to the next one.
98
99Variable definitions are a single name followed by equal followed by a list
100of names, possibly empty.
101
102Rule headers are a nonempty list of names, followed by a colon, followed by
103another list of names, possibly empty. Basically, the syntax of a rule is as
104follows:
105
106@verbatim
107targets : prerequisites
108 shell script
109@endverbatim
110
111List of names are space-separated sequences of names. If a name contains
112a space character, it should be put into double quotes. Names cannot be
113any of the following special characters `:$(),="`. Again, quotation
114should be used. Quotation marks can be escaped by a backslash inside
115quoted names.
116
117\subsection sec-variables Variables
118
119Variables can be used to factor lists of targets or prerequisites. They are
120expanded as they are encountered during <b>Remakefile</b> parsing.
121
122@verbatim
123VAR2 = a
124VAR1 = c d
125VAR2 += $(VAR1) b
126$(VAR2) e :
127@endverbatim
128
129Variable assignments can appear instead of prerequisites inside non-generic
130rules with no script. They are then expanded inside the corresponding
131generic rule.
132
133@verbatim
134foo.o: CFLAGS += -DBAR
135
136%.o : %.c
137 gcc $(CFLAGS) -MMD -MF $@.d -o $@ -c $<
138 remake -r < $@.d
139 rm $@.d
140@endverbatim
141
142Note: contrarily to <b>make</b>, variable names have to be enclosed in
143parentheses. For instance, `$y` is not a shorthand for <tt>\$(y)</tt> and
144is left unexpanded.
145
146\subsection sec-autovars Automatic variables
147
148The following special symbols can appear inside scripts:
149
150- `$<` expands to the first static prerequisite of the rule.
151- `$^` expands to all the static prerequisites of the rule, including
152 duplicates if any.
153- `$@` expands to the first target of the rule.
154- `$*` expands to the string that matched `%` in a generic rule.
155- `$$` expands to a single dollar symbol.
156
157Note: contrarily to <b>make</b>, there are no corresponding variables.
158For instance, `$^` is not a shorthand for `$(^)`. Another difference is
159that `$@` is always the first target, not the one that triggered the
160rule.
161
162\subsection sec-functions Built-in functions
163
164<b>remake</b> also supports a few built-in functions inspired from <b>make</b>.
165
166- <tt>$(addprefix <i>prefix</i>, <i>list</i>)</tt> returns the list obtained
167 by prepending its first argument to each element of its second argument.
168- <tt>$(addsuffix <i>suffix</i>, <i>list</i>)</tt> returns the list obtained
169 by appending its first argument to each element of its second argument.
170
171\subsection sec-order Order-only prerequisites
172
173If the static prerequisites of a rule contain a pipe symbol, prerequisites
174on its right do not cause the targets to become obsolete if they are newer
175(unless they are also dynamically registered as dependencies). They are
176meant to be used when the targets do not directly depend on them, but the
177computation of their dynamic dependencies does.
178
179@verbatim
180%.o : %.c | parser.h
181 gcc -MMD -MF $@.d -o $@ -c $<
182 remake -r < $@.d
183 rm $@.d
184
185parser.c parser.h: parser.y
186 yacc -d -o parser.c parser.y
187@endverbatim
188
189\subsection sec-static-pattern Static pattern rules
190
191A rule with the following structure is expanded into several rules, one
192per target.
193
194@verbatim
195targets: pattern1 pattern2 ...: prerequisites
196@endverbatim
197
198Every target is matched against one of the patterns containing the `%`
199character. A rule is then created using the patterns as targets, after
200having substituted `%` in the patterns and prerequisites. The automatic
201variable `$*` can be used in the script of the rule.
202
203\subsection sec-special-tgt Special targets
204
205Target `.PHONY` marks its prerequisites as being always obsolete.
206
207\subsection sec-special-var Special variables
208
209Variable `.OPTIONS` is handled specially. Its content enables some
210features of <b>remake</b> that are not enabled by default.
211
212- `variable-propagation`: When a variable is set in the prerequisite
213 part of a rule, it is propagated to the rules of all the targets this rule
214 depends on. Note that, as in <b>make</b>, this feature introduces
215 non-determinism: the content of some variables will depend on the build order.
216
217\section sec-semantics Semantics
218
219\subsection src-obsolete When are targets obsolete?
220
221A target is obsolete:
222
223- if there is no file corresponding to the target, or to one of its siblings
224 in a multi-target rule,
225- if any of its dynamic prerequisites from a previous run or any of its static
226 prerequisites is obsolete,
227- if the latest file corresponding to its siblings or itself is older than any
228 of its dynamic prerequisites or static prerequisites.
229
230In all the other cases, it is assumed to be up-to-date (and so are all its
231siblings). Note that the last rule above says "latest" and not "earliest". While
232it might cause some obsolete targets to go unnoticed in corner cases, it allows
233for the following kind of rules:
234
235@verbatim
236config.h stamp-config_h: config.h.in config.status
237 ./config.status config.h
238 touch stamp-config_h
239@endverbatim
240
241A `config.status` file generally does not update header files (here
242`config.h`) if they would not change. As a consequence, if not for the
243`stamp-config_h` file above, a header would always be considered obsolete
244once one of its prerequisites is modified. Note that touching `config.h`
245rather than `stamp-config_h` would defeat the point of not updating it in
246the first place, since the program files would need to be rebuilt.
247
248Once all the static prerequisites of a target have been rebuilt, <b>remake</b>
249checks whether the target still needs to be built. If it was obsolete only
250because its prerequisites needed to be rebuilt and none of them changed, the
251target is assumed to be up-to-date.
252
253\subsection sec-rules How are targets (re)built?
254
255There are two kinds of rules. If any of the targets or prerequisites contains
256a `%` character, the rule is said to be <em>generic</em>. All the
257targets of the rule shall then contain a single `%` character. All the
258other rules are said to be <em>specific</em>.
259
260A rule is said to <em>match</em> a given target:
261
262- if it is specific and the target appears inside its target list,
263- if it is generic and there is a way to replace the `%` character
264 from one of its targets so that it matches the given target.
265
266When <b>remake</b> tries to build a given target, it looks for a specific rule
267that matches it. If there is one and its script is nonempty, it uses it to
268rebuild the target.
269
270Otherwise, it looks for a generic rule that matches the target. If there are
271several matching rules, it chooses the one with the shortest pattern (and if
272there are several ones, the earliest one). It then looks for specific rules
273that match each target of the generic rule. All the prerequisites of these
274specific rules are added to those of the generic rule. The script of the
275generic rule is used to build the target.
276
277Example:
278
279@verbatim
280t%1 t2%: p1 p%2
281 commands building t%1 and t2%
282
283t2z: p4
284 commands building t2z
285
286ty1: p3
287
288# t2x is built by the first rule (which also builds tx1) and its prerequisites are p1, px2
289# t2y is built by the first rule (which also builds ty1) and its prerequisites are p1, py2, p3
290# t2z is built by the second rule and its prerequisite is p4
291@endverbatim
292
293The set of rules from <b>Remakefile</b> is ill-formed:
294
295- if any specific rule matching a target of the generic rule has a nonempty script,
296- if any target of the generic rule is matched by a generic rule with a shorter pattern.
297
298\section sec-compilation Compilation
299
300- On Linux, MacOSX, and BSD: `g++ -o remake remake.cpp`
301- On Windows: `g++ -o remake.exe remake.cpp -lws2_32`
302
303Installing <b>remake</b> is needed only if <b>Remakefile</b> does not
304specify the path to the executable for its recursive calls. Thanks to its
305single source file, <b>remake</b> can be shipped inside other packages and
306built at configuration time.
307
308\section sec-differences Differences with other build systems
309
310Differences with <b>make</b>:
311
312- Dynamic dependencies are supported.
313- For rules with multiple targets, the shell script is executed only once
314 and is assumed to build all the targets. There is no need for
315 convoluted rules that are robust enough for parallel builds. For generic
316 rules, this is similar to the behavior of pattern rules from <b>gmake</b>.
317- As with <b>redo</b>, only one shell is run when executing a script,
318 rather than one per script line. Note that the shells are run with
319 option `-e`, thus causing them to exit as soon as an error is
320 encountered.
321- The prerequisites of generic rules (known as implicit rules in <b>make</b>
322 lingo) are not used to decide between several of them, which means that
323 <b>remake</b> does not select one for which it could satisfy the dependencies.
324- Variables and built-in functions are expanded as they are encountered
325 during <b>Remakefile</b> parsing.
326- Target-specific variables are not propagated, unless specifically enabled,
327 since this causes non-deterministic builds.
328
329Differences with <b>redo</b>:
330
331- As with <b>make</b>, it is possible to write the following kind of rules
332 in <b>remake</b>.
333@verbatim
334Remakefile: Remakefile.in ./config.status
335 ./config.status Remakefile
336@endverbatim
337- If a target is already built the first time <b>remake</b> runs, it still
338 uses the static prerequisites of rules mentioning it to check whether it
339 needs to be rebuilt. It does not assume it to be up-to-date. As with
340 <b>redo</b> though, if its obsolete status would be due to a dynamic
341 prerequisite, it will go unnoticed; it should be removed beforehand.
342- Multiple targets are supported.
343- <b>remake</b> has almost no features: no checksum-based dependencies, no
344 compatibility with job servers, etc.
345
346\section sec-limitations Limitations
347
348- If a rule script calls <b>remake</b>, the current working directory should
349 be the directory containing <b>Remakefile</b> (or the working directory
350 from the original <b>remake</b> if it was called with option `-f`).
351- As with <b>make</b>, variables passed on the command line should keep
352 the same values, to ensure deterministic builds.
353- Some cases of ill-formed rules are not caught by <b>remake</b> and can
354 thus lead to unpredictable behaviors.
355
356\section sec-links Links
357
358@see http://cr.yp.to/redo.html for the philosophy of <b>redo</b> and
359https://github.com/apenwarr/redo for an implementation and some comprehensive documentation.
360
361\section sec-licensing Licensing
362
363@author Guillaume Melquiond
364@version 0.16
365@date 2012-2024
366@copyright
367This program is free software: you can redistribute it and/or modify
368it under the terms of the GNU General Public License as published by
369the Free Software Foundation, either version 3 of the License, or
370(at your option) any later version.
371\n
372This program is distributed in the hope that it will be useful,
373but WITHOUT ANY WARRANTY; without even the implied warranty of
374MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
375GNU General Public License for more details.
376
377\section sec-internals Internals
378
379The parent <b>remake</b> process acts as a server. The other ones have a
380REMAKE_SOCKET environment variable that tells them how to contact the
381server. They send the content of the REMAKE_JOB_ID environment variable,
382so that the server can associate the child targets to the jobs that
383spawned them. They then wait for completion and exit with the status
384returned by the server. This is handled by #client_mode.
385
386The server calls #load_dependencies and #save_dependencies to serialize
387dynamic dependencies from <b>.remake</b>. It loads <b>Remakefile</b> with
388#load_rules. It then runs #server_mode, which calls #server_loop.
389
390When building a target, the following sequence of events happens:
391
392- #start calls #find_rule (and #find_generic_rule) to get the rule.
393- It then creates a pseudo-client if the rule has static dependencies, or
394 calls #run_script otherwise. In both cases, a new job is created; the
395 rule and the variables are stored into #jobs.
396- #run_script creates a shell process and stores it in #job_pids. It
397 increases #running_jobs.
398- The child process possibly calls <b>remake</b> with a list of targets.
399- #accept_client receives a build request from a child process and adds
400 it to #clients. It also records the new dependencies of the job into
401 #dependencies. It increases #waiting_jobs.
402- #handle_clients uses #get_status to look up the obsoleteness of the
403 targets.
404- Once the targets of a request have been built or one of them has failed,
405 #handle_clients calls #complete_request and removes the request from
406 #clients.
407- If the build targets come from a pseudo-client, #complete_request calls
408 #run_script. Otherwise it sends the reply to the corresponding child
409 process and decreases #waiting_jobs.
410- When a child process ends, #server_loop calls #finalize_job, which
411 removes the process from #job_pids, decreases #running_jobs, and calls
412 #complete_job.
413- #complete_job removes the job from #jobs and calls #update_status
414 to change the status of the targets. It also removes the target files in
415 case of failure.
416*/
417
418#ifdef _WIN32
419#define WIN32_LEAN_AND_MEAN
420#define WINDOWS
421#endif
422
423#include <fstream>
424#include <iostream>
425#include <list>
426#include <map>
427#include <set>
428#include <sstream>
429#include <string>
430#include <vector>
431#include <cassert>
432#include <cstdlib>
433#include <cstring>
434#include <ctime>
435#include <errno.h>
436#include <fcntl.h>
437#include <signal.h>
438#include <unistd.h>
439#include <sys/stat.h>
440#include <sys/types.h>
441
442#ifdef __APPLE__
443#define MACOSX
444#endif
445
446#ifdef __linux__
447#define LINUX
448#endif
449
450#ifdef WINDOWS
451#include <windows.h>
452#include <winbase.h>
453#include <winsock2.h>
454#define pid_t HANDLE
455typedef SOCKET socket_t;
456#else
457#include <sys/socket.h>
458#include <sys/un.h>
459#include <sys/wait.h>
460typedef int socket_t;
461enum { INVALID_SOCKET = -1 };
462extern char **environ;
463#endif
464
465#if (defined(WINDOWS) || defined(MACOSX)) && !defined(MSG_NOSIGNAL)
466enum { MSG_NOSIGNAL = 0 };
467#endif
468
469typedef std::list<std::string> string_list;
470
471typedef std::set<std::string> string_set;
472
473/**
474 * Reference-counted shared object.
475 * @note The default constructor delays the creation of the object until it
476 * is first dereferenced.
477 */
478template<class T>
480{
481 struct content
482 {
483 size_t cnt;
485 content(): cnt(1) {}
486 content(T const &t): cnt(1), val(t) {}
487 };
488 mutable content *ptr;
489 ref_ptr(): ptr(NULL) {}
490 ref_ptr(T const &t): ptr(new content(t)) {}
491 ref_ptr(ref_ptr const &p): ptr(p.ptr) { if (ptr) ++ptr->cnt; }
492 ~ref_ptr() { if (ptr && --ptr->cnt == 0) delete ptr; }
494 {
495 if (ptr == p.ptr) return *this;
496 if (ptr && --ptr->cnt == 0) delete ptr;
497 ptr = p.ptr;
498 if (ptr) ++ptr->cnt;
499 return *this;
500 }
501 T &operator*() const
502 {
503 if (!ptr) ptr = new content;
504 return ptr->val;
505 }
506 T *operator->() const { return &**this; }
507};
508
514
515typedef std::map<std::string, ref_ptr<dependency_t> > dependency_map;
516
517typedef std::map<std::string, string_list> variable_map;
518
519/**
520 * Build status of a target.
521 */
523{
524 Uptodate, ///< Target is up-to-date.
525 Todo, ///< Target is missing or obsolete.
526 Recheck, ///< Target has an obsolete dependency.
527 Running, ///< Target is being rebuilt.
528 RunningRecheck, ///< Static prerequisites are being rebuilt.
529 Remade, ///< Target was successfully rebuilt.
530 Failed ///< Build failed for target.
532
533/**
534 * Build status of a target.
535 */
537{
538 status_e status; ///< Actual status.
539 time_t last; ///< Last-modified date.
540};
541
542typedef std::map<std::string, status_t> status_map;
543
544/**
545 * Delayed assignment to a variable.
546 */
548{
549 bool append;
551};
552
553typedef std::map<std::string, assign_t> assign_map;
554
555/**
556 * A rule loaded from Remakefile.
557 */
558struct rule_t
559{
560 string_list targets; ///< Files produced by this rule.
561 string_list deps; ///< Dependencies used for an implicit call to remake at the start of the script.
562 string_list wdeps; ///< Like #deps, except that they are not registered as dependencies.
563 assign_map assigns; ///< Assignment of variables.
564 std::string stem; ///< Stem used to instantiate the rule, if any.
565 std::string script; ///< Shell script for building the targets.
566};
567
568typedef std::list<rule_t> rule_list;
569
570typedef std::map<std::string, ref_ptr<rule_t> > rule_map;
571
572/**
573 * A job created from a set of rules.
574 */
575
576struct job_t
577{
578 rule_t rule; ///< Original rule.
579 variable_map vars; ///< Values of local variables.
580};
581
582typedef std::map<int, job_t> job_map;
583
584typedef std::map<pid_t, int> pid_job_map;
585
586/**
587 * Client waiting for a request to complete.
588 *
589 * There are two kinds of clients:
590 * - real clients, which are instances of remake created by built scripts,
591 * - pseudo clients, which are created by the server to build specific targets.
592 *
593 * Among pseudo clients, there are two categories:
594 * - original clients, which are created for the targets passed on the
595 * command line by the user or for the initial regeneration of the rule file,
596 * - dependency clients, which are created to handle rules that have
597 * explicit dependencies and thus to emulate a call to remake.
598 */
600{
601 socket_t socket; ///< Socket used to reply to the client (invalid for pseudo clients).
602 int job_id; ///< Job for which the built script called remake and spawned the client (negative for original clients).
603 bool failed; ///< Whether some targets failed in mode -k.
604 string_list pending; ///< Targets not yet started.
605 string_set running; ///< Targets being built.
606 variable_map vars; ///< Variables set on request.
607 bool delayed; ///< Whether it is a dependency client and a script has to be started on request completion.
608 client_t(): socket(INVALID_SOCKET), job_id(-1), failed(false), delayed(false) {}
609};
610
611typedef std::list<client_t> client_list;
612
613/**
614 * Map from variable names to their content.
615 * Initialized with the values passed on the command line.
616 */
618
619/**
620 * Map from targets to their known dependencies.
621 */
623
624/**
625 * Map from targets to their build status.
626 */
628
629/**
630 * Set of generic rules loaded from Remakefile.
631 */
633
634/**
635 * Map from targets to specific rules loaded from Remakefile.
636 */
638
639/**
640 * Map of jobs being built.
641 */
643
644/**
645 * Map from jobs to shell pids.
646 */
648
649/**
650 * List of clients waiting for a request to complete.
651 * New clients are put to front, so that the build process is depth-first.
652 */
654
655/**
656 * Maximum number of parallel jobs (non-positive if unbounded).
657 * Can be modified by the -j option.
658 */
659static int max_active_jobs = 1;
660
661/**
662 * Whether to keep building targets in case of failure.
663 * Can be modified by the -k option.
664 */
665static bool keep_going = false;
666
667/**
668 * Number of jobs currently running:
669 * - it increases when a process is created in #run_script,
670 * - it decreases when a completion message is received in #finalize_job.
671 *
672 * @note There might be some jobs running while #clients is empty.
673 * Indeed, if a client requested two targets to be rebuilt, if they
674 * are running concurrently, if one of them fails, the client will
675 * get a failure notice and might terminate before the other target
676 * finishes.
677 */
678static int running_jobs = 0;
679
680/**
681 * Number of jobs currently waiting for a build request to finish:
682 * - it increases when a build request is received in #accept_client
683 * (since the client is presumably waiting for the reply),
684 * - it decreases when a reply is sent in #complete_request.
685 */
686static int waiting_jobs = 0;
687
688/**
689 * Global counter used to produce increasing job numbers.
690 * @see jobs
691 */
692static int job_counter = 0;
693
694/**
695 * Socket on which the server listens for client request.
696 */
698
699/**
700 * Whether the request of an original client failed.
701 */
702static bool build_failure;
703
704#ifndef WINDOWS
705/**
706 * Name of the server socket in the file system.
707 */
708static char *socket_name;
709#endif
710
711/**
712 * Name of the first target of the first specific rule, used for default run.
713 */
714static std::string first_target;
715
716/**
717 * Whether a short message should be displayed for each target.
718 */
719static bool show_targets = true;
720
721/**
722 * Whether script commands are echoed.
723 */
724static bool echo_scripts = false;
725
726/**
727 * Time at the start of the program.
728 */
729static time_t now = time(NULL);
730
731/**
732 * Directory with respect to which command-line names are relative.
733 */
734static std::string working_dir;
735
736/**
737 * Directory with respect to which targets are relative.
738 */
739static std::string prefix_dir;
740
741/**
742 * Whether the prefix directory is different from #working_dir.
743 */
745
746/**
747 * Whether target-specific variables are propagated to prerequisites.
748 */
749static bool propagate_vars = false;
750
751/**
752 * Whether targets are unconditionally obsolete.
753 */
754static bool obsolete_targets = false;
755
756#ifndef WINDOWS
757static sigset_t old_sigmask;
758
759static volatile sig_atomic_t got_SIGCHLD = 0;
760
761static void sigchld_handler(int)
762{
763 got_SIGCHLD = 1;
764}
765
766static void sigint_handler(int)
767{
768 // Child processes will receive the signal too, so just prevent
769 // new jobs from starting and wait for the running jobs to fail.
770 keep_going = false;
771}
772#endif
773
774struct log
775{
777 int depth;
778 log(): active(false), open(false), depth(0)
779 {
780 }
781 std::ostream &operator()()
782 {
783 if (open) std::cerr << std::endl;
784 assert(depth >= 0);
785 std::cerr << std::string(depth * 2, ' ');
786 open = false;
787 return std::cerr;
788 }
789 std::ostream &operator()(bool o)
790 {
791 if (o && open) std::cerr << std::endl;
792 if (!o) --depth;
793 assert(depth >= 0);
794 if (o || !open) std::cerr << std::string(depth * 2, ' ');
795 if (o) ++depth;
796 open = o;
797 return std::cerr;
798 }
799};
800
801static struct log debug;
802
804{
807 {
808 }
810 {
811 if (debug.active && still_open) debug(false) << "done\n";
812 }
813};
814
815#define DEBUG if (debug.active) debug()
816#define DEBUG_open log_auto_close auto_close; if (debug.active) debug(true)
817#define DEBUG_close if ((auto_close.still_open = false), debug.active) debug(false)
818
819/**
820 * Strong typedef for strings that need escaping.
821 * @note The string is stored as a reference, so the constructed object is
822 * meant to be immediately consumed.
823 */
825{
826 std::string const &input;
827 escape_string(std::string const &s): input(s) {}
828};
829
830/**
831 * Write the string in @a se to @a out if it does not contain any special
832 * characters, a quoted and escaped string otherwise.
833 */
834static std::ostream &operator<<(std::ostream &out, escape_string const &se)
835{
836 std::string const &s = se.input;
837 char const *quoted_char = ",: '";
838 char const *escaped_char = "\"\\$!";
839 bool need_quotes = false;
840 char *buf = NULL;
841 size_t len = s.length(), last = 0, j = 0;
842 for (size_t i = 0; i < len; ++i)
843 {
844 if (strchr(escaped_char, s[i]))
845 {
846 need_quotes = true;
847 if (!buf) buf = new char[len * 2];
848 memcpy(&buf[j], &s[last], i - last);
849 j += i - last;
850 buf[j++] = '\\';
851 buf[j++] = s[i];
852 last = i + 1;
853 }
854 if (!need_quotes && strchr(quoted_char, s[i]))
855 need_quotes = true;
856 }
857 if (!need_quotes) return out << s;
858 out << '"';
859 if (!buf) return out << s << '"';
860 out.write(buf, j);
861 out.write(&s[last], len - last);
862 delete[] buf;
863 return out << '"';
864}
865
866/**
867 * @defgroup paths Path helpers
868 *
869 * @{
870 */
871
872/**
873 * Initialize #working_dir.
874 */
875static void init_working_dir()
876{
877 char buf[1024];
878 char *res = getcwd(buf, sizeof(buf));
879 if (!res)
880 {
881 perror("Failed to get working directory");
882 exit(EXIT_FAILURE);
883 }
884 working_dir = buf;
885#ifdef WINDOWS
886 for (size_t i = 0, l = working_dir.size(); i != l; ++i)
887 {
888 if (working_dir[i] == '\\') working_dir[i] = '/';
889 }
890#endif
892}
893
894/**
895 * Initialize #prefix_dir and switch to it.
896 */
897static void init_prefix_dir()
898{
899 for (;;)
900 {
901 struct stat s;
902 if (stat((prefix_dir + "/Remakefile").c_str(), &s) == 0)
903 {
904 if (!changed_prefix_dir) return;
905 if (chdir(prefix_dir.c_str()))
906 {
907 perror("Failed to change working directory");
908 exit(EXIT_FAILURE);
909 }
910 if (show_targets)
911 {
912 std::cout << "remake: Entering directory `" << prefix_dir << '\'' << std::endl;
913 }
914 return;
915 }
916 size_t pos = prefix_dir.find_last_of('/');
917 if (pos == std::string::npos)
918 {
919 std::cerr << "Failed to locate Remakefile in the current directory or one of its parents" << std::endl;
920 exit(EXIT_FAILURE);
921 }
922 prefix_dir.erase(pos);
923 changed_prefix_dir = true;
924 }
925}
926
927/**
928 * Normalize an absolute path with respect to @a p.
929 * Paths outside the subtree are left unchanged.
930 */
931static std::string normalize_abs(std::string const &s, std::string const &p)
932{
933 size_t l = p.length();
934 if (s.compare(0, l, p)) return s;
935 size_t ll = s.length();
936 if (ll == l) return ".";
937 if (s[l] != '/')
938 {
939 size_t pos = s.rfind('/', l);
940 assert(pos != std::string::npos);
941 return s.substr(pos + 1);
942 }
943 if (ll == l + 1) return ".";
944 return s.substr(l + 1);
945}
946
947/**
948 * Normalize path @a s (possibly relative to @a w) with respect to @a p.
949 *
950 * - If both @a p and @a w are empty, the function just removes ".", "..", "//".
951 * - If only @a p is empty, the function returns an absolute path.
952 */
953static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
954{
955#ifdef WINDOWS
956 char const *delim = "/\\";
957#else
958 char delim = '/';
959#endif
960 size_t pos = s.find_first_of(delim);
961 if (pos == std::string::npos && w == p) return s;
962 bool absolute = pos == 0;
963 if (!absolute && w != p && !w.empty())
964 return normalize(w + '/' + s, w, p);
965 size_t prev = 0, len = s.length();
966 string_list l;
967 for (;;)
968 {
969 if (pos != prev)
970 {
971 std::string n = s.substr(prev, pos - prev);
972 if (n == "..")
973 {
974 if (!l.empty()) l.pop_back();
975 else if (!absolute && !w.empty())
976 return normalize(w + '/' + s, w, p);
977 }
978 else if (n != ".")
979 l.push_back(n);
980 }
981 ++pos;
982 if (pos >= len) break;
983 prev = pos;
984 pos = s.find_first_of(delim, prev);
985 if (pos == std::string::npos) pos = len;
986 }
987 string_list::const_iterator i = l.begin(), i_end = l.end();
988 if (i == i_end) return absolute ? "/" : ".";
989 std::string n;
990 if (absolute) n.push_back('/');
991 n.append(*i);
992 for (++i; i != i_end; ++i)
993 {
994 n.push_back('/');
995 n.append(*i);
996 }
997 if (absolute && !p.empty()) return normalize_abs(n, p);
998 return n;
999}
1000
1001/**
1002 * Normalize the content of a list of targets.
1003 */
1004static void normalize_list(string_list &l, std::string const &w, std::string const &p)
1005{
1006 for (string_list::iterator i = l.begin(),
1007 i_end = l.end(); i != i_end; ++i)
1008 {
1009 *i = normalize(*i, w, p);
1010 }
1011}
1012
1013/** @} */
1014
1015/**
1016 * @defgroup lexer Lexer
1017 *
1018 * @{
1019 */
1020
1021/**
1022 * Skip spaces.
1023 */
1024static void skip_spaces(std::istream &in)
1025{
1026 char c;
1027 while (strchr(" \t", (c = in.get()))) {}
1028 if (in.good()) in.putback(c);
1029}
1030
1031/**
1032 * Skip empty lines.
1033 */
1034static void skip_empty(std::istream &in)
1035{
1036 char c;
1037 while (strchr("\r\n", (c = in.get()))) {}
1038 if (in.good()) in.putback(c);
1039}
1040
1041/**
1042 * Skip end of line. If @a multi is true, skip the following empty lines too.
1043 * @return true if there was a line to end.
1044 */
1045static bool skip_eol(std::istream &in, bool multi = false)
1046{
1047 char c = in.get();
1048 if (c == '\r') c = in.get();
1049 if (c != '\n' && in.good()) in.putback(c);
1050 if (c != '\n' && !in.eof()) return false;
1051 if (multi) skip_empty(in);
1052 return true;
1053}
1054
1055enum
1056{
1058 Word = 1 << 1,
1059 Colon = 1 << 2,
1060 Equal = 1 << 3,
1061 Dollarpar = 1 << 4,
1062 Rightpar = 1 << 5,
1063 Comma = 1 << 6,
1064 Plusequal = 1 << 7,
1065 Pipe = 1 << 8,
1066};
1067
1068/**
1069 * Skip spaces and peek at the next token.
1070 * If it is one of @a mask, skip it (if it is not Word) and return it.
1071 * @note For composite tokens allowed by @a mask, input characters might
1072 * have been eaten even for an Unexpected result.
1073 */
1074static int expect_token(std::istream &in, int mask)
1075{
1076 while (true)
1077 {
1078 skip_spaces(in);
1079 char c = in.peek();
1080 if (!in.good()) return Unexpected;
1081 int tok;
1082 switch (c)
1083 {
1084 case '\r':
1085 case '\n': return Unexpected;
1086 case ':': tok = Colon; break;
1087 case ',': tok = Comma; break;
1088 case '=': tok = Equal; break;
1089 case ')': tok = Rightpar; break;
1090 case '|': tok = Pipe; break;
1091 case '$':
1092 if (!(mask & Dollarpar)) return Unexpected;
1093 in.ignore(1);
1094 tok = Dollarpar;
1095 if (in.peek() != '(') return Unexpected;
1096 break;
1097 case '+':
1098 if (!(mask & Plusequal)) return Unexpected;
1099 in.ignore(1);
1100 tok = Plusequal;
1101 if (in.peek() != '=') return Unexpected;
1102 break;
1103 case '\\':
1104 in.ignore(1);
1105 if (skip_eol(in)) continue;
1106 in.putback('\\');
1107 return mask & Word ? Word : Unexpected;
1108 default:
1109 return mask & Word ? Word : Unexpected;
1110 }
1111 if (!(tok & mask)) return Unexpected;
1112 in.ignore(1);
1113 return tok;
1114 }
1115}
1116
1117/**
1118 * Read a (possibly quoted) word.
1119 */
1120static std::string read_word(std::istream &in, bool detect_equal = true)
1121{
1122 int c = in.peek();
1123 std::string res;
1124 if (!in.good()) return res;
1125 char const *separators = " \t\r\n$(),:";
1126 bool quoted = c == '"';
1127 if (quoted) in.ignore(1);
1128 bool plus = false;
1129 while (true)
1130 {
1131 c = in.peek();
1132 if (!in.good()) return res;
1133 if (quoted)
1134 {
1135 in.ignore(1);
1136 if (c == '\\')
1137 res += in.get();
1138 else if (c == '"')
1139 quoted = false;
1140 else
1141 res += c;
1142 continue;
1143 }
1144 if (detect_equal && c == '=')
1145 {
1146 if (plus) in.putback('+');
1147 return res;
1148 }
1149 if (plus)
1150 {
1151 res += '+';
1152 plus = false;
1153 }
1154 if (strchr(separators, c)) return res;
1155 in.ignore(1);
1156 if (detect_equal && c == '+') plus = true;
1157 else res += c;
1158 }
1159}
1160
1161/** @} */
1162
1163/**
1164 * @defgroup stream Token streams
1165 *
1166 * @{
1167 */
1168
1169/**
1170 * Possible results from word producers.
1171 */
1178
1179/**
1180 * Interface for word producers.
1181 */
1183{
1184 virtual ~generator() {}
1185 virtual input_status next(std::string &) = 0;
1186};
1187
1188/**
1189 * Generator for the words of a variable.
1190 */
1192{
1193 std::string name;
1194 string_list::const_iterator vcur, vend;
1195 variable_generator(std::string const &, variable_map const *);
1196 input_status next(std::string &);
1197};
1198
1200 variable_map const *local_variables): name(n)
1201{
1202 if (local_variables)
1203 {
1204 variable_map::const_iterator i = local_variables->find(name);
1205 if (i != local_variables->end())
1206 {
1207 vcur = i->second.begin();
1208 vend = i->second.end();
1209 return;
1210 }
1211 }
1212 variable_map::const_iterator i = variables.find(name);
1213 if (i == variables.end()) return;
1214 vcur = i->second.begin();
1215 vend = i->second.end();
1216}
1217
1219{
1220 if (vcur != vend)
1221 {
1222 res = *vcur;
1223 ++vcur;
1224 return Success;
1225 }
1226 return Eof;
1227}
1228
1229/**
1230 * Generator for the words of an input stream.
1231 */
1233{
1234 std::istream &in;
1238 input_generator(std::istream &i, variable_map const *lv, bool e = false)
1239 : in(i), nested(NULL), local_variables(lv), earliest_exit(e), done(false) {}
1240 input_status next(std::string &);
1241 ~input_generator() { assert(!nested); }
1242};
1243
1244static generator *get_function(input_generator const &, std::string const &);
1245
1247{
1248 if (nested)
1249 {
1250 restart:
1251 input_status s = nested->next(res);
1252 if (s == Success) return Success;
1253 delete nested;
1254 nested = NULL;
1255 if (s == SyntaxError) return SyntaxError;
1256 }
1257 if (done) return Eof;
1258 if (earliest_exit) done = true;
1259 switch (expect_token(in, Word | Dollarpar))
1260 {
1261 case Word:
1262 res = read_word(in, false);
1263 return Success;
1264 case Dollarpar:
1265 {
1266 std::string name = read_word(in, false);
1267 if (name.empty()) return SyntaxError;
1268 if (expect_token(in, Rightpar))
1270 else
1271 {
1272 nested = get_function(*this, name);
1273 if (!nested) return SyntaxError;
1274 }
1275 goto restart;
1276 }
1277 default:
1278 return Eof;
1279 }
1280}
1281
1282/**
1283 * Read a list of words from an input generator.
1284 * @return false if a syntax error was encountered.
1285 */
1287{
1288 while (true)
1289 {
1290 res.push_back(std::string());
1291 input_status s = in.next(res.back());
1292 if (s == Success) continue;
1293 res.pop_back();
1294 return s == Eof;
1295 }
1296}
1297
1298static bool read_words(std::istream &in, string_list &res)
1299{
1300 input_generator gen(in, NULL);
1301 return read_words(gen, res);
1302}
1303
1304/**
1305 * Generator for the result of function addprefix.
1306 */
1308{
1311 string_list::const_iterator prei;
1312 size_t prej, prel;
1313 std::string suf;
1314 addprefix_generator(input_generator const &, bool &);
1315 input_status next(std::string &);
1316};
1317
1319 : gen(top.in, top.local_variables)
1320{
1321 if (!read_words(gen, pre)) return;
1322 if (!expect_token(gen.in, Comma)) return;
1323 prej = 0;
1324 prel = pre.size();
1325 ok = true;
1326}
1327
1329{
1330 if (prej)
1331 {
1332 produce:
1333 if (prej == prel)
1334 {
1335 res = *prei + suf;
1336 prej = 0;
1337 }
1338 else
1339 {
1340 res = *prei++;
1341 ++prej;
1342 }
1343 return Success;
1344 }
1345 switch (gen.next(res))
1346 {
1347 case Success:
1348 if (!prel) return Success;
1349 prei = pre.begin();
1350 prej = 1;
1351 suf = res;
1352 goto produce;
1353 case Eof:
1355 default:
1356 return SyntaxError;
1357 }
1358}
1359
1360/**
1361 * Generator for the result of function addsuffix.
1362 */
1364{
1367 string_list::const_iterator sufi;
1368 size_t sufj, sufl;
1369 std::string pre;
1370 addsuffix_generator(input_generator const &, bool &);
1371 input_status next(std::string &);
1372};
1373
1375 : gen(top.in, top.local_variables)
1376{
1377 if (!read_words(gen, suf)) return;
1378 if (!expect_token(gen.in, Comma)) return;
1379 sufj = 0;
1380 sufl = suf.size();
1381 ok = true;
1382}
1383
1385{
1386 if (sufj)
1387 {
1388 if (sufj != sufl)
1389 {
1390 res = *sufi++;
1391 ++sufj;
1392 return Success;
1393 }
1394 sufj = 0;
1395 }
1396 switch (gen.next(res))
1397 {
1398 case Success:
1399 if (!sufl) return Success;
1400 sufi = suf.begin();
1401 sufj = 1;
1402 res += *sufi++;
1403 return Success;
1404 case Eof:
1406 default:
1407 return SyntaxError;
1408 }
1409}
1410
1411/**
1412 * Return a generator for function @a name.
1413 */
1414static generator *get_function(input_generator const &in, std::string const &name)
1415{
1416 skip_spaces(in.in);
1417 generator *g = NULL;
1418 bool ok = false;
1419 if (name == "addprefix") g = new addprefix_generator(in, ok);
1420 else if (name == "addsuffix") g = new addsuffix_generator(in, ok);
1421 if (!g || ok) return g;
1422 delete g;
1423 return NULL;
1424}
1425
1426/** @} */
1427
1428/**
1429 * @defgroup database Dependency database
1430 *
1431 * @{
1432 */
1433
1434/**
1435 * Load dependencies from @a in.
1436 */
1437static void load_dependencies(std::istream &in)
1438{
1439 if (false)
1440 {
1441 error:
1442 std::cerr << "Failed to load database" << std::endl;
1443 exit(EXIT_FAILURE);
1444 }
1445
1446 while (!in.eof())
1447 {
1448 string_list targets;
1449 if (!read_words(in, targets)) goto error;
1450 if (in.eof()) return;
1451 if (targets.empty()) goto error;
1452 DEBUG << "reading dependencies of target " << targets.front() << std::endl;
1453 if (in.get() != ':') goto error;
1455 dep->targets = targets;
1456 string_list deps;
1457 if (!read_words(in, deps)) goto error;
1458 dep->deps.insert(deps.begin(), deps.end());
1459 for (string_list::const_iterator i = targets.begin(),
1460 i_end = targets.end(); i != i_end; ++i)
1461 {
1462 dependencies[*i] = dep;
1463 }
1464 skip_empty(in);
1465 }
1466}
1467
1468/**
1469 * Load known dependencies from file `.remake`.
1470 */
1472{
1473 DEBUG_open << "Loading database... ";
1474 std::ifstream in(".remake");
1475 if (!in.good())
1476 {
1477 DEBUG_close << "not found\n";
1478 return;
1479 }
1481}
1482
1483
1484/**
1485 * Save all the dependencies in file `.remake`.
1486 */
1488{
1489 DEBUG_open << "Saving database... ";
1490 std::ofstream db(".remake");
1491 while (!dependencies.empty())
1492 {
1493 ref_ptr<dependency_t> dep = dependencies.begin()->second;
1494 for (string_list::const_iterator i = dep->targets.begin(),
1495 i_end = dep->targets.end(); i != i_end; ++i)
1496 {
1497 db << escape_string(*i) << ' ';
1498 dependencies.erase(*i);
1499 }
1500 db << ':';
1501 for (string_set::const_iterator i = dep->deps.begin(),
1502 i_end = dep->deps.end(); i != i_end; ++i)
1503 {
1504 db << ' ' << escape_string(*i);
1505 }
1506 db << std::endl;
1507 }
1508}
1509
1510/** @} */
1511
1512static void merge_rule(rule_t &dest, rule_t const &src);
1513static void instantiate_rule(std::string const &target, rule_t const &src, rule_t &dst);
1514
1515/**
1516 * @defgroup parser Rule parser
1517 *
1518 * @{
1519 */
1520
1521/**
1522 * Register a specific rule with an empty script:
1523 *
1524 * - Check that none of the targets already has an associated rule with a
1525 * nonempty script.
1526 * - Create a new rule with a single target for each target, if needed.
1527 * - Add the prerequisites of @a rule to all these associated rules.
1528 */
1529static void register_transparent_rule(rule_t const &rule, string_list const &targets)
1530{
1531 assert(rule.script.empty());
1532 for (string_list::const_iterator i = targets.begin(),
1533 i_end = targets.end(); i != i_end; ++i)
1534 {
1535 std::pair<rule_map::iterator, bool> j =
1536 specific_rules.insert(std::make_pair(*i, ref_ptr<rule_t>()));
1537 ref_ptr<rule_t> &r = j.first->second;
1538 if (j.second)
1539 {
1540 r = ref_ptr<rule_t>(rule);
1541 r->targets = string_list(1, *i);
1542 continue;
1543 }
1544 if (!r->script.empty())
1545 {
1546 std::cerr << "Failed to load rules: " << *i
1547 << " cannot be the target of several rules" << std::endl;
1549 }
1550 assert(r->targets.size() == 1 && r->targets.front() == *i);
1551 merge_rule(*r, rule);
1552 }
1553
1554 for (string_list::const_iterator i = targets.begin(),
1555 i_end = targets.end(); i != i_end; ++i)
1556 {
1558 if (dep->targets.empty()) dep->targets.push_back(*i);
1559 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1560 }
1561}
1562
1563/**
1564 * Register a specific rule with a nonempty script:
1565 *
1566 * - Check that none of the targets already has an associated rule.
1567 * - Create a single shared rule and associate it to all the targets.
1568 * - Merge the prerequisites of all the targets into a single set and
1569 * add the prerequisites of the rule to it. (The preexisting
1570 * prerequisites, if any, come from a previous run.)
1571 */
1572static void register_scripted_rule(rule_t const &rule)
1573{
1574 ref_ptr<rule_t> r(rule);
1575 for (string_list::const_iterator i = rule.targets.begin(),
1576 i_end = rule.targets.end(); i != i_end; ++i)
1577 {
1578 std::pair<rule_map::iterator, bool> j =
1579 specific_rules.insert(std::make_pair(*i, r));
1580 if (j.second) continue;
1581 std::cerr << "Failed to load rules: " << *i
1582 << " cannot be the target of several rules" << std::endl;
1584 }
1585
1587 dep->targets = rule.targets;
1588 dep->deps.insert(rule.deps.begin(), rule.deps.end());
1589 for (string_list::const_iterator i = rule.targets.begin(),
1590 i_end = rule.targets.end(); i != i_end; ++i)
1591 {
1593 dep->deps.insert(d->deps.begin(), d->deps.end());
1594 d = dep;
1595 }
1596}
1597
1598/**
1599 * Register a specific rule.
1600 */
1601static void register_rule(rule_t const &rule)
1602{
1603 if (!rule.script.empty())
1604 {
1606 }
1607 else
1608 {
1609 // Swap away the targets to avoid costly copies when registering.
1610 rule_t &r = const_cast<rule_t &>(rule);
1611 string_list targets;
1612 targets.swap(r.targets);
1613 register_transparent_rule(r, targets);
1614 targets.swap(r.targets);
1615 }
1616
1617 // If there is no default target yet, mark it as such.
1618 if (first_target.empty())
1619 first_target = rule.targets.front();
1620}
1621
1622/**
1623 * Read a rule starting with target @a first, if nonempty.
1624 * Store into #generic_rules or #specific_rules depending on its genericity.
1625 */
1626static void load_rule(std::istream &in, std::string const &first)
1627{
1628 DEBUG_open << "Reading rule for target " << first << "... ";
1629 if (false)
1630 {
1631 error:
1632 DEBUG_close << "failed\n";
1633 std::cerr << "Failed to load rules: syntax error" << std::endl;
1635 }
1636
1637 // Read targets and check genericity.
1638 string_list targets;
1639 if (!read_words(in, targets)) goto error;
1640 if (!first.empty()) targets.push_front(first);
1641 else if (targets.empty()) goto error;
1642 else DEBUG << "actual target: " << targets.front() << std::endl;
1643 bool generic = false;
1644 normalize_list(targets, "", "");
1645 for (string_list::const_iterator i = targets.begin(),
1646 i_end = targets.end(); i != i_end; ++i)
1647 {
1648 if (i->empty()) goto error;
1649 if ((i->find('%') != std::string::npos) != generic)
1650 {
1651 if (i == targets.begin()) generic = true;
1652 else goto error;
1653 }
1654 }
1655 skip_spaces(in);
1656 if (in.get() != ':') goto error;
1657
1658 bool assignment = false, static_pattern = false;
1659
1660 rule_t rule;
1661 rule.targets.swap(targets);
1662
1663 // Read dependencies.
1664 {
1665 string_list v;
1666 if (expect_token(in, Word))
1667 {
1668 std::string d = read_word(in);
1669 if (int tok = expect_token(in, Equal | Plusequal))
1670 {
1671 if (!read_words(in, v)) goto error;
1672 assign_t &a = rule.assigns[d];
1673 a.append = tok == Plusequal;
1674 a.value.swap(v);
1675 assignment = true;
1676 goto end_line;
1677 }
1678 v.push_back(d);
1679 }
1680
1681 if (!read_words(in, v)) goto error;
1682 normalize_list(v, "", "");
1683 rule.deps.swap(v);
1684
1685 if (expect_token(in, Colon))
1686 {
1687 if (!read_words(in, v)) goto error;
1688 normalize_list(v, "", "");
1689 targets.swap(rule.targets);
1690 rule.targets.swap(rule.deps);
1691 rule.deps.swap(v);
1692 if (rule.targets.empty()) goto error;
1693 for (string_list::const_iterator i = rule.targets.begin(),
1694 i_end = rule.targets.end(); i != i_end; ++i)
1695 {
1696 if (i->find('%') == std::string::npos) goto error;
1697 }
1698 generic = false;
1699 static_pattern = true;
1700 }
1701
1702 if (expect_token(in, Pipe))
1703 {
1704 if (!read_words(in, v)) goto error;
1705 normalize_list(v, "", "");
1706 rule.wdeps.swap(v);
1707 }
1708 }
1709
1710 end_line:
1711 skip_spaces(in);
1712 if (!skip_eol(in, true)) goto error;
1713
1714 // Read script.
1715 std::ostringstream buf;
1716 while (true)
1717 {
1718 char c = in.get();
1719 if (!in.good()) break;
1720 if (c == '\t' || c == ' ')
1721 {
1722 in.get(*buf.rdbuf());
1723 if (in.fail() && !in.eof()) in.clear();
1724 }
1725 else if (c == '\r' || c == '\n')
1726 buf << c;
1727 else
1728 {
1729 in.putback(c);
1730 break;
1731 }
1732 }
1733 rule.script = buf.str();
1734
1735 // Register phony targets.
1736 if (rule.targets.front() == ".PHONY")
1737 {
1738 for (string_list::const_iterator i = rule.deps.begin(),
1739 i_end = rule.deps.end(); i != i_end; ++i)
1740 {
1741 status[*i].status = Todo;
1742 }
1743 return;
1744 }
1745
1746 // Add generic rules to the correct set.
1747 if (generic)
1748 {
1749 if (assignment) goto error;
1750 generic_rules.push_back(rule);
1751 return;
1752 }
1753
1754 if (!static_pattern)
1755 {
1756 if (!rule.script.empty() && assignment) goto error;
1757 register_rule(rule);
1758 return;
1759 }
1760
1761 for (string_list::const_iterator i = targets.begin(),
1762 i_end = targets.end(); i != i_end; ++i)
1763 {
1764 rule_t r;
1765 instantiate_rule(*i, rule, r);
1766 if (!r.stem.empty()) register_rule(r);
1767 }
1768}
1769
1770/**
1771 * Load rules from @a remakefile.
1772 * If some rules have dependencies and non-generic targets, add these
1773 * dependencies to the targets.
1774 */
1775static void load_rules(std::string const &remakefile)
1776{
1777 DEBUG_open << "Loading rules... ";
1778 if (false)
1779 {
1780 error:
1781 std::cerr << "Failed to load rules: syntax error" << std::endl;
1783 }
1784 std::ifstream in(remakefile.c_str());
1785 if (!in.good())
1786 {
1787 std::cerr << "Failed to load rules: no Remakefile found" << std::endl;
1789 }
1790 skip_empty(in);
1791
1793
1794 // Read rules
1795 while (in.good())
1796 {
1797 char c = in.peek();
1798 if (c == '#')
1799 {
1800 while (in.get() != '\n') {}
1801 skip_empty(in);
1802 continue;
1803 }
1804 if (c == ' ' || c == '\t') goto error;
1805 if (expect_token(in, Word))
1806 {
1807 std::string name = read_word(in);
1808 if (name.empty()) goto error;
1809 if (int tok = expect_token(in, Equal | Plusequal))
1810 {
1811 DEBUG << "Assignment to variable " << name << std::endl;
1812 string_list value;
1813 if (!read_words(in, value)) goto error;
1814 string_list &dest =
1815 *(name == ".OPTIONS" ? &options : &variables[name]);
1816 if (tok == Equal) dest.swap(value);
1817 else dest.splice(dest.end(), value);
1818 if (!skip_eol(in, true)) goto error;
1819 }
1820 else load_rule(in, name);
1821 }
1822 else load_rule(in, std::string());
1823 }
1824
1825 // Set actual options.
1826 for (string_list::const_iterator i = options.begin(),
1827 i_end = options.end(); i != i_end; ++i)
1828 {
1829 if (*i == "variable-propagation") propagate_vars = true;
1830 else
1831 {
1832 std::cerr << "Failed to load rules: unrecognized option" << std::endl;
1834 }
1835 }
1836}
1837
1838/** @} */
1839
1840/**
1841 * @defgroup rules Rule resolution
1842 *
1843 * @{
1844 */
1845
1846static void merge_rule(rule_t &dest, rule_t const &src)
1847{
1848 dest.deps.insert(dest.deps.end(), src.deps.begin(), src.deps.end());
1849 dest.wdeps.insert(dest.wdeps.end(), src.wdeps.begin(), src.wdeps.end());
1850 for (assign_map::const_iterator i = src.assigns.begin(),
1851 i_end = src.assigns.end(); i != i_end; ++i)
1852 {
1853 if (!i->second.append)
1854 {
1855 new_assign:
1856 dest.assigns[i->first] = i->second;
1857 continue;
1858 }
1859 assign_map::iterator j = dest.assigns.find(i->first);
1860 if (j == dest.assigns.end()) goto new_assign;
1861 j->second.value.insert(j->second.value.end(),
1862 i->second.value.begin(), i->second.value.end());
1863 }
1864}
1865
1866/**
1867 * Substitute a pattern into a list of strings.
1868 */
1869static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
1870{
1871 for (string_list::const_iterator i = src.begin(),
1872 i_end = src.end(); i != i_end; ++i)
1873 {
1874 size_t pos = i->find('%');
1875 if (pos == std::string::npos) dst.push_back(*i);
1876 else dst.push_back(i->substr(0, pos) + pat + i->substr(pos + 1));
1877 }
1878}
1879
1880/**
1881 * Instantiate a specific rule, given a target and a generic rule.
1882 * If the rule @a dst already contains a stem longer than the one found,
1883 * it is left unchanged.
1884 */
1885static void instantiate_rule(std::string const &target, rule_t const &src, rule_t &dst)
1886{
1887 size_t tlen = target.length(), plen = dst.stem.length();
1888 for (string_list::const_iterator j = src.targets.begin(),
1889 j_end = src.targets.end(); j != j_end; ++j)
1890 {
1891 size_t len = j->length();
1892 if (tlen < len) continue;
1893 if (plen && plen <= tlen - (len - 1)) continue;
1894 size_t pos = j->find('%');
1895 if (pos == std::string::npos) continue;
1896 size_t len2 = len - (pos + 1);
1897 if (j->compare(0, pos, target, 0, pos) ||
1898 j->compare(pos + 1, len2, target, tlen - len2, len2))
1899 continue;
1900 plen = tlen - (len - 1);
1901 dst = rule_t();
1902 dst.stem = target.substr(pos, plen);
1903 dst.script = src.script;
1904 substitute_pattern(dst.stem, src.targets, dst.targets);
1905 substitute_pattern(dst.stem, src.deps, dst.deps);
1906 substitute_pattern(dst.stem, src.wdeps, dst.wdeps);
1907 break;
1908 }
1909}
1910
1911/**
1912 * Find a generic rule matching @a target:
1913 * - the one leading to shorter matches has priority,
1914 * - among equivalent rules, the earliest one has priority.
1915 */
1916static void find_generic_rule(job_t &job, std::string const &target)
1917{
1918 for (rule_list::const_iterator i = generic_rules.begin(),
1919 i_end = generic_rules.end(); i != i_end; ++i)
1920 {
1921 instantiate_rule(target, *i, job.rule);
1922 }
1923}
1924
1925/**
1926 * Find a specific rule matching @a target. Return a generic one otherwise.
1927 * If there is both a specific rule with an empty script and a generic rule, the
1928 * generic one is returned after adding the dependencies of the specific one.
1929 */
1930static void find_rule(job_t &job, std::string const &target)
1931{
1932 rule_map::const_iterator i = specific_rules.find(target),
1933 i_end = specific_rules.end();
1934 // If there is a specific rule with a script, return it.
1935 if (i != i_end && !i->second->script.empty())
1936 {
1937 job.rule = *i->second;
1938 return;
1939 }
1941 // If there is no generic rule, return the specific rule (no script), if any.
1942 if (job.rule.targets.empty())
1943 {
1944 if (i != i_end)
1945 {
1946 job.rule = *i->second;
1947 return;
1948 }
1949 }
1950 // Optimize the lookup when there is only one target (already looked up).
1951 if (job.rule.targets.size() == 1)
1952 {
1953 if (i == i_end) return;
1954 merge_rule(job.rule, *i->second);
1955 return;
1956 }
1957 // Add the dependencies of the specific rules of every target to the
1958 // generic rule. If any of those rules has a nonempty script, error out.
1959 for (string_list::const_iterator j = job.rule.targets.begin(),
1960 j_end = job.rule.targets.end(); j != j_end; ++j)
1961 {
1962 i = specific_rules.find(*j);
1963 if (i == i_end) continue;
1964 if (!i->second->script.empty()) return;
1965 merge_rule(job.rule, *i->second);
1966 }
1967}
1968
1969/** @} */
1970
1971/**
1972 * @defgroup status Target status
1973 *
1974 * @{
1975 */
1976
1977/**
1978 * Compute and memoize the status of @a target:
1979 * - if the file does not exist, the target is obsolete,
1980 * - if any dependency is obsolete or younger than the file, it is obsolete,
1981 * - otherwise it is up-to-date.
1982 *
1983 * @note For rules with multiple targets, all the targets share the same
1984 * status. (If one is obsolete, they all are.) The second rule above
1985 * is modified in that case: the latest target is chosen, not the oldest!
1986 */
1987static status_t const &get_status(std::string const &target)
1988{
1989 std::pair<status_map::iterator,bool> i =
1990 status.insert(std::make_pair(target, status_t()));
1991 status_t &ts = i.first->second;
1992 if (!i.second) return ts;
1993 DEBUG_open << "Checking status of " << target << "... ";
1994 dependency_map::const_iterator j = dependencies.find(target);
1995 if (j == dependencies.end())
1996 {
1997 struct stat s;
1998 if (stat(target.c_str(), &s) != 0)
1999 {
2000 DEBUG_close << "missing\n";
2001 ts.status = Todo;
2002 ts.last = 0;
2003 return ts;
2004 }
2005 DEBUG_close << "up-to-date\n";
2006 ts.status = Uptodate;
2007 ts.last = s.st_mtime;
2008 return ts;
2009 }
2010 if (obsolete_targets)
2011 {
2012 DEBUG_close << "forcefully obsolete\n";
2013 ts.status = Todo;
2014 ts.last = 0;
2015 return ts;
2016 }
2017 dependency_t const &dep = *j->second;
2019 time_t latest = 0;
2020 for (string_list::const_iterator k = dep.targets.begin(),
2021 k_end = dep.targets.end(); k != k_end; ++k)
2022 {
2023 struct stat s;
2024 if (stat(k->c_str(), &s) != 0)
2025 {
2026 if (st == Uptodate) DEBUG_close << *k << " missing\n";
2027 s.st_mtime = 0;
2028 st = Todo;
2029 }
2030 status[*k].last = s.st_mtime;
2031 if (s.st_mtime > latest) latest = s.st_mtime;
2032 }
2033 if (st != Uptodate) goto update;
2034 for (string_set::const_iterator k = dep.deps.begin(),
2035 k_end = dep.deps.end(); k != k_end; ++k)
2036 {
2037 status_t const &ts_ = get_status(*k);
2038 if (latest < ts_.last)
2039 {
2040 DEBUG_close << "older than " << *k << std::endl;
2041 st = Todo;
2042 goto update;
2043 }
2044 if (ts_.status != Uptodate && st != Recheck)
2045 {
2046 DEBUG << "obsolete dependency " << *k << std::endl;
2047 st = Recheck;
2048 }
2049 }
2050 if (st == Uptodate) DEBUG_close << "all siblings up-to-date\n";
2051 update:
2052 for (string_list::const_iterator k = dep.targets.begin(),
2053 k_end = dep.targets.end(); k != k_end; ++k)
2054 {
2055 status[*k].status = st;
2056 }
2057 return ts;
2058}
2059
2060/**
2061 * Change the status of @a target to #Remade or #Uptodate depending on whether
2062 * its modification time changed.
2063 */
2064static void update_status(std::string const &target)
2065{
2066 DEBUG_open << "Rechecking status of " << target << "... ";
2067 status_map::iterator i = status.find(target);
2068 assert(i != status.end());
2069 status_t &ts = i->second;
2070 ts.status = Remade;
2071 if (ts.last >= now)
2072 {
2073 DEBUG_close << "possibly remade\n";
2074 return;
2075 }
2076 struct stat s;
2077 if (stat(target.c_str(), &s) != 0)
2078 {
2079 DEBUG_close << "missing\n";
2080 ts.last = 0;
2081 }
2082 else if (s.st_mtime != ts.last)
2083 {
2084 DEBUG_close << "remade\n";
2085 ts.last = s.st_mtime;
2086 }
2087 else
2088 {
2089 DEBUG_close << "unchanged\n";
2090 ts.status = Uptodate;
2091 }
2092}
2093
2094/**
2095 * Check whether all the prerequisites of @a target ended being up-to-date.
2096 */
2097static bool still_need_rebuild(std::string const &target)
2098{
2099 status_map::const_iterator i = status.find(target);
2100 assert(i != status.end());
2101 if (i->second.status != RunningRecheck) return true;
2102 DEBUG_open << "Rechecking obsoleteness of " << target << "... ";
2103 dependency_map::const_iterator j = dependencies.find(target);
2104 assert(j != dependencies.end());
2105 dependency_t const &dep = *j->second;
2106 for (string_set::const_iterator k = dep.deps.begin(),
2107 k_end = dep.deps.end(); k != k_end; ++k)
2108 {
2109 if (status[*k].status != Uptodate) return true;
2110 }
2111 for (string_list::const_iterator k = dep.targets.begin(),
2112 k_end = dep.targets.end(); k != k_end; ++k)
2113 {
2114 status[*k].status = Uptodate;
2115 }
2116 DEBUG_close << "no longer obsolete\n";
2117 return false;
2118}
2119
2120/** @} */
2121
2122/**
2123 * @defgroup server Server
2124 *
2125 * @{
2126 */
2127
2128/**
2129 * Handle job completion.
2130 */
2131static void complete_job(int job_id, bool success, bool started = true)
2132{
2133 DEBUG << "Completing job " << job_id << '\n';
2134 job_map::iterator i = jobs.find(job_id);
2135 assert(i != jobs.end());
2136 string_list const &targets = i->second.rule.targets;
2137 if (success)
2138 {
2139 bool show = show_targets && started;
2140 if (show) std::cout << "Finished";
2141 for (string_list::const_iterator j = targets.begin(),
2142 j_end = targets.end(); j != j_end; ++j)
2143 {
2144 update_status(*j);
2145 if (show) std::cout << ' ' << *j;
2146 }
2147 if (show) std::cout << std::endl;
2148 }
2149 else
2150 {
2151 std::cerr << "Failed to build";
2152 for (string_list::const_iterator j = targets.begin(),
2153 j_end = targets.end(); j != j_end; ++j)
2154 {
2155 std::cerr << ' ' << *j;
2156 update_status(*j);
2157 status_e &s = status[*j].status;
2158 if (s != Uptodate)
2159 {
2160 DEBUG << "Removing " << *j << '\n';
2161 remove(j->c_str());
2162 }
2163 s = Failed;
2164 }
2165 std::cerr << std::endl;
2166 }
2167 jobs.erase(i);
2168}
2169
2170/**
2171 * Return the script obtained by substituting variables.
2172 */
2173static std::string prepare_script(job_t const &job)
2174{
2175 std::string const &s = job.rule.script;
2176 std::istringstream in(s);
2177 std::ostringstream out;
2178 size_t len = s.size();
2179
2180 while (!in.eof())
2181 {
2182 size_t pos = in.tellg(), p = s.find('$', pos);
2183 if (p == std::string::npos || p == len - 1) p = len;
2184 out.write(&s[pos], p - pos);
2185 if (p == len) break;
2186 ++p;
2187 switch (s[p])
2188 {
2189 case '$':
2190 out << '$';
2191 in.seekg(p + 1);
2192 break;
2193 case '<':
2194 if (!job.rule.deps.empty())
2195 out << job.rule.deps.front();
2196 in.seekg(p + 1);
2197 break;
2198 case '^':
2199 {
2200 bool first = true;
2201 for (string_list::const_iterator i = job.rule.deps.begin(),
2202 i_end = job.rule.deps.end(); i != i_end; ++i)
2203 {
2204 if (first) first = false;
2205 else out << ' ';
2206 out << *i;
2207 }
2208 in.seekg(p + 1);
2209 break;
2210 }
2211 case '@':
2212 assert(!job.rule.targets.empty());
2213 out << job.rule.targets.front();
2214 in.seekg(p + 1);
2215 break;
2216 case '*':
2217 out << job.rule.stem;
2218 in.seekg(p + 1);
2219 break;
2220 case '(':
2221 {
2222 in.seekg(p - 1);
2223 bool first = true;
2224 input_generator gen(in, &job.vars, true);
2225 while (true)
2226 {
2227 std::string w;
2228 input_status s = gen.next(w);
2229 if (s == SyntaxError)
2230 {
2231 // TODO
2232 return "false";
2233 }
2234 if (s == Eof) break;
2235 if (first) first = false;
2236 else out << ' ';
2237 out << w;
2238 }
2239 break;
2240 }
2241 default:
2242 // Let dollars followed by an unrecognized character
2243 // go through. This differs from Make, which would
2244 // use a one-letter variable.
2245 out << '$';
2246 in.seekg(p);
2247 }
2248 }
2249
2250 return out.str();
2251}
2252
2253/**
2254 * Execute the script from @a rule.
2255 */
2256static status_e run_script(int job_id, job_t const &job)
2257{
2259 dep->targets = job.rule.targets;
2260 dep->deps.insert(job.rule.deps.begin(), job.rule.deps.end());
2261 if (show_targets) std::cout << "Building";
2262 for (string_list::const_iterator i = job.rule.targets.begin(),
2263 i_end = job.rule.targets.end(); i != i_end; ++i)
2264 {
2265 dependencies[*i] = dep;
2266 if (show_targets) std::cout << ' ' << *i;
2267 }
2268 if (show_targets) std::cout << std::endl;
2269
2270 std::string script = prepare_script(job);
2271
2272 std::ostringstream job_id_buf;
2273 job_id_buf << job_id;
2274 std::string job_id_ = job_id_buf.str();
2275
2276 DEBUG_open << "Starting script for job " << job_id << "... ";
2277 if (script.empty())
2278 {
2279 DEBUG_close << "no script\n";
2280 complete_job(job_id, true);
2281 return Remade;
2282 }
2283
2284 if (false)
2285 {
2286 error:
2287 DEBUG_close << "failed\n";
2288 complete_job(job_id, false);
2289 return Failed;
2290 }
2291
2292#ifdef WINDOWS
2293 HANDLE pfd[2];
2294 if (false)
2295 {
2296 error2:
2297 CloseHandle(pfd[0]);
2298 CloseHandle(pfd[1]);
2299 goto error;
2300 }
2301 if (!CreatePipe(&pfd[0], &pfd[1], NULL, 0))
2302 goto error;
2304 goto error2;
2306 ZeroMemory(&si, sizeof(STARTUPINFO));
2307 si.cb = sizeof(STARTUPINFO);
2308 si.hStdError = GetStdHandle(STD_ERROR_HANDLE);
2309 si.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
2310 si.hStdInput = pfd[0];
2311 si.dwFlags |= STARTF_USESTDHANDLES;
2314 if (!SetEnvironmentVariable("REMAKE_JOB_ID", job_id_.c_str()))
2315 goto error2;
2316 char const *argv = echo_scripts ? "SH.EXE -e -s -v" : "SH.EXE -e -s";
2317 if (!CreateProcess(NULL, (char *)argv, NULL, NULL,
2318 true, 0, NULL, NULL, &si, &pi))
2319 {
2320 goto error2;
2321 }
2322 CloseHandle(pi.hThread);
2323 DWORD len = script.length(), wlen;
2324 if (!WriteFile(pfd[1], script.c_str(), len, &wlen, NULL) || wlen < len)
2325 std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2326 CloseHandle(pfd[0]);
2327 CloseHandle(pfd[1]);
2328 ++running_jobs;
2329 job_pids[pi.hProcess] = job_id;
2330 return Running;
2331#else
2332 int pfd[2];
2333 if (false)
2334 {
2335 error2:
2336 close(pfd[0]);
2337 close(pfd[1]);
2338 goto error;
2339 }
2340 if (pipe(pfd) == -1)
2341 goto error;
2342 if (setenv("REMAKE_JOB_ID", job_id_.c_str(), 1))
2343 goto error2;
2344 if (pid_t pid = vfork())
2345 {
2346 if (pid == -1) goto error2;
2347 ssize_t len = script.length();
2348 if (write(pfd[1], script.c_str(), len) < len)
2349 std::cerr << "Unexpected failure while sending script to shell" << std::endl;
2350 close(pfd[0]);
2351 close(pfd[1]);
2352 ++running_jobs;
2353 job_pids[pid] = job_id;
2354 return Running;
2355 }
2356 // Child process starts here. Notice the use of vfork above.
2357 char const *argv[5] = { "sh", "-e", "-s", NULL, NULL };
2358 if (echo_scripts) argv[3] = "-v";
2359 close(pfd[1]);
2360 if (pfd[0] != 0)
2361 {
2362 dup2(pfd[0], 0);
2363 close(pfd[0]);
2364 }
2366 execve("/bin/sh", (char **)argv, environ);
2368#endif
2369}
2370
2371/**
2372 * Create a job for @a target according to the loaded rules.
2373 * Mark all the targets from the rule as running and reset their dependencies.
2374 * Inherit variables from @a current, if enabled.
2375 * If the rule has dependencies, create a new client to build them just
2376 * before @a current, and change @a current so that it points to it.
2377 */
2378static status_e start(std::string const &target, client_list::iterator &current)
2379{
2380 int job_id = job_counter++;
2381 DEBUG_open << "Starting job " << job_id << " for " << target << "... ";
2382 job_t &job = jobs[job_id];
2384 if (job.rule.targets.empty())
2385 {
2386 status[target].status = Failed;
2387 DEBUG_close << "failed\n";
2388 std::cerr << "No rule for building " << target << std::endl;
2389 return Failed;
2390 }
2391 bool has_deps = !job.rule.deps.empty() || !job.rule.wdeps.empty();
2393 if (has_deps && status[target].status == Recheck)
2395 for (string_list::const_iterator i = job.rule.targets.begin(),
2396 i_end = job.rule.targets.end(); i != i_end; ++i)
2397 {
2398 status[*i].status = st;
2399 }
2400 if (propagate_vars) job.vars = current->vars;
2401 for (assign_map::const_iterator i = job.rule.assigns.begin(),
2402 i_end = job.rule.assigns.end(); i != i_end; ++i)
2403 {
2404 std::pair<variable_map::iterator, bool> k =
2405 job.vars.insert(std::make_pair(i->first, string_list()));
2406 string_list &v = k.first->second;
2407 if (i->second.append)
2408 {
2409 if (k.second)
2410 {
2411 variable_map::const_iterator j = variables.find(i->first);
2412 if (j != variables.end()) v = j->second;
2413 }
2414 }
2415 else if (!k.second) v.clear();
2416 v.insert(v.end(), i->second.value.begin(), i->second.value.end());
2417 }
2418 if (has_deps)
2419 {
2420 current = clients.insert(current, client_t());
2421 current->job_id = job_id;
2422 current->pending = job.rule.deps;
2423 current->pending.insert(current->pending.end(),
2424 job.rule.wdeps.begin(), job.rule.wdeps.end());
2425 if (propagate_vars) current->vars = job.vars;
2426 current->delayed = true;
2427 return RunningRecheck;
2428 }
2429 return run_script(job_id, job);
2430}
2431
2432/**
2433 * Send a reply to a client then remove it.
2434 * If the client was a dependency client, start the actual script.
2435 */
2436static void complete_request(client_t &client, bool success)
2437{
2438 DEBUG_open << "Completing request from client of job " << client.job_id << "... ";
2439 if (client.delayed)
2440 {
2441 assert(client.socket == INVALID_SOCKET);
2442 if (success)
2443 {
2444 job_map::const_iterator i = jobs.find(client.job_id);
2445 assert(i != jobs.end());
2446 if (still_need_rebuild(i->second.rule.targets.front()))
2447 run_script(client.job_id, i->second);
2448 else complete_job(client.job_id, true, false);
2449 }
2450 else complete_job(client.job_id, false);
2451 }
2452 else if (client.socket != INVALID_SOCKET)
2453 {
2454 char res = success ? 1 : 0;
2455 send(client.socket, &res, 1, MSG_NOSIGNAL);
2456 #ifdef WINDOWS
2457 closesocket(client.socket);
2458 #else
2459 close(client.socket);
2460 #endif
2461 --waiting_jobs;
2462 }
2463
2464 if (client.job_id < 0 && !success) build_failure = true;
2465}
2466
2467/**
2468 * Return whether there are slots for starting new jobs.
2469 */
2470static bool has_free_slots()
2471{
2472 if (max_active_jobs <= 0) return true;
2474}
2475
2476/**
2477 * Handle client requests:
2478 * - check for running targets that have finished,
2479 * - start as many pending targets as allowed,
2480 * - complete the request if there are neither running nor pending targets
2481 * left or if any of them failed.
2482 *
2483 * @return true if some child processes are still running.
2484 *
2485 * @post If there are pending requests, at least one child process is running.
2486 *
2487 * @invariant New free slots cannot appear during a run, since the only way to
2488 * decrease #running_jobs is #finalize_job and the only way to
2489 * increase #waiting_jobs is #accept_client. None of these functions
2490 * are called during a run. So breaking out as soon as there are no
2491 * free slots left is fine.
2492 */
2493static bool handle_clients()
2494{
2495 DEBUG_open << "Handling client requests... ";
2496 restart:
2497 bool need_restart = false;
2498
2499 for (client_list::iterator i = clients.begin(), i_next = i,
2500 i_end = clients.end(); i != i_end; i = i_next)
2501 {
2502 if (!has_free_slots()) break;
2503 ++i_next;
2504 DEBUG_open << "Handling client from job " << i->job_id << "... ";
2505
2506 // Remove running targets that have finished.
2507 for (string_set::iterator j = i->running.begin(), j_next = j,
2508 j_end = i->running.end(); j != j_end; j = j_next)
2509 {
2510 ++j_next;
2511 status_map::const_iterator k = status.find(*j);
2512 assert(k != status.end());
2513 switch (k->second.status)
2514 {
2515 case Running:
2516 case RunningRecheck:
2517 break;
2518 case Failed:
2519 i->failed = true;
2520 if (!keep_going) goto complete;
2521 // fallthrough
2522 case Uptodate:
2523 case Remade:
2524 i->running.erase(j);
2525 break;
2526 case Recheck:
2527 case Todo:
2528 assert(false);
2529 }
2530 }
2531
2532 // Start pending targets.
2533 while (!i->pending.empty())
2534 {
2535 std::string target = i->pending.front();
2536 i->pending.pop_front();
2537 switch (get_status(target).status)
2538 {
2539 case Running:
2540 case RunningRecheck:
2541 i->running.insert(target);
2542 break;
2543 case Failed:
2545 i->failed = true;
2546 if (!keep_going) goto complete;
2547 // fallthrough
2548 case Uptodate:
2549 case Remade:
2550 break;
2551 case Recheck:
2552 case Todo:
2553 client_list::iterator j = i;
2554 switch (start(target, i))
2555 {
2556 case Failed:
2557 goto pending_failed;
2558 case Running:
2559 // A shell was started, check for free slots.
2560 j->running.insert(target);
2561 if (!has_free_slots()) return true;
2562 break;
2563 case RunningRecheck:
2564 // Switch to the dependency client that was inserted.
2565 j->running.insert(target);
2566 i_next = j;
2567 break;
2568 case Remade:
2569 // Nothing to run.
2570 need_restart = true;
2571 break;
2572 default:
2573 assert(false);
2574 }
2575 }
2576 }
2577
2578 // Try to complete the request.
2579 // (This might start a new job if it was a dependency client.)
2580 if (i->running.empty() || i->failed)
2581 {
2582 complete:
2583 complete_request(*i, !i->failed);
2584 DEBUG_close << (i->failed ? "failed\n" : "finished\n");
2585 clients.erase(i);
2586 need_restart = true;
2587 }
2588 }
2589
2590 if (running_jobs != waiting_jobs) return true;
2591 if (running_jobs == 0 && clients.empty()) return false;
2592 if (need_restart) goto restart;
2593
2594 // There is a circular dependency.
2595 // Try to break it by completing one of the requests.
2596 assert(!clients.empty());
2597 std::cerr << "Circular dependency detected" << std::endl;
2598 client_list::iterator i = clients.begin();
2599 complete_request(*i, false);
2600 clients.erase(i);
2601 goto restart;
2602}
2603
2604/**
2605 * Create a named unix socket that listens for build requests. Also set
2606 * the REMAKE_SOCKET environment variable that will be inherited by all
2607 * the job scripts.
2608 */
2609static void create_server()
2610{
2611 if (false)
2612 {
2613 error:
2614 perror("Failed to create server");
2615#ifndef WINDOWS
2616 error2:
2617#endif
2619 }
2620 DEBUG_open << "Creating server... ";
2621
2622#ifdef WINDOWS
2623 // Prepare a windows socket.
2624 struct sockaddr_in socket_addr;
2625 socket_addr.sin_family = AF_INET;
2626 socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2627 socket_addr.sin_port = 0;
2628
2629 // Create and listen to the socket.
2630 socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2631 if (socket_fd == INVALID_SOCKET) goto error;
2633 goto error;
2634 if (bind(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2635 goto error;
2636 int len = sizeof(sockaddr_in);
2637 if (getsockname(socket_fd, (struct sockaddr *)&socket_addr, &len))
2638 goto error;
2639 std::ostringstream buf;
2640 buf << socket_addr.sin_port;
2641 if (!SetEnvironmentVariable("REMAKE_SOCKET", buf.str().c_str()))
2642 goto error;
2643 if (listen(socket_fd, 1000)) goto error;
2644#else
2645 // Set signal handlers for SIGCHLD and SIGINT.
2646 // Block SIGCHLD (unblocked during select).
2650 if (sigprocmask(SIG_BLOCK, &sigmask, &old_sigmask) == -1) goto error;
2651 struct sigaction sa;
2652 sa.sa_flags = 0;
2653 sigemptyset(&sa.sa_mask);
2654 sa.sa_handler = &sigchld_handler;
2655 if (sigaction(SIGCHLD, &sa, NULL) == -1) goto error;
2656 sa.sa_handler = &sigint_handler;
2657 if (sigaction(SIGINT, &sa, NULL) == -1) goto error;
2658
2659 // Prepare a named unix socket in temporary directory.
2660 socket_name = tempnam(NULL, "rmk-");
2661 if (!socket_name) goto error2;
2662 struct sockaddr_un socket_addr;
2663 size_t len = strlen(socket_name);
2664 if (len >= sizeof(socket_addr.sun_path) - 1) goto error2;
2665 socket_addr.sun_family = AF_UNIX;
2666 strcpy(socket_addr.sun_path, socket_name);
2667 len += sizeof(socket_addr.sun_family);
2668 if (setenv("REMAKE_SOCKET", socket_name, 1)) goto error;
2669
2670 // Create and listen to the socket.
2671#ifdef LINUX
2672 socket_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
2673 if (socket_fd == INVALID_SOCKET) goto error;
2674#else
2675 socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2676 if (socket_fd == INVALID_SOCKET) goto error;
2677 if (fcntl(socket_fd, F_SETFD, FD_CLOEXEC) < 0) goto error;
2678#endif
2679 if (bind(socket_fd, (struct sockaddr *)&socket_addr, len))
2680 goto error;
2681 if (listen(socket_fd, 1000)) goto error;
2682#endif
2683}
2684
2685/**
2686 * Accept a connection from a client, get the job it spawned from,
2687 * get the targets, and mark them as dependencies of the job targets.
2688 */
2689static void accept_client()
2690{
2691 DEBUG_open << "Handling client request... ";
2692
2693 // Accept connection.
2694#ifdef WINDOWS
2696 if (fd == INVALID_SOCKET) return;
2698 {
2699 error2:
2700 std::cerr << "Unexpected failure while setting connection with client" << std::endl;
2701 closesocket(fd);
2702 return;
2703 }
2704 // WSAEventSelect puts sockets into nonblocking mode, so disable it here.
2705 u_long nbio = 0;
2706 if (ioctlsocket(fd, FIONBIO, &nbio)) goto error2;
2707#elif defined(LINUX)
2709 if (fd < 0) return;
2710#else
2711 int fd = accept(socket_fd, NULL, NULL);
2712 if (fd < 0) return;
2713 if (fcntl(fd, F_SETFD, FD_CLOEXEC) < 0) return;
2714#endif
2715 clients.push_front(client_t());
2716 client_list::iterator proc = clients.begin();
2717
2718 if (false)
2719 {
2720 error:
2721 DEBUG_close << "failed\n";
2722 std::cerr << "Received an ill-formed client message" << std::endl;
2723 #ifdef WINDOWS
2724 closesocket(fd);
2725 #else
2726 close(fd);
2727 #endif
2728 clients.erase(proc);
2729 return;
2730 }
2731
2732 // Receive message. Stop when encountering two nuls in a row.
2733 std::vector<char> buf;
2734 size_t len = 0;
2735 while (len < sizeof(int) + 2 || buf[len - 1] || buf[len - 2])
2736 {
2737 buf.resize(len + 1024);
2738 ssize_t l = recv(fd, &buf[0] + len, 1024, 0);
2739 if (l <= 0) goto error;
2740 len += l;
2741 }
2742
2743 // Parse job that spawned the client.
2744 int job_id;
2745 memcpy(&job_id, &buf[0], sizeof(int));
2746 proc->socket = fd;
2747 proc->job_id = job_id;
2748 job_map::const_iterator i = jobs.find(job_id);
2749 if (i == jobs.end()) goto error;
2750 DEBUG << "receiving request from job " << job_id << std::endl;
2751 if (propagate_vars) proc->vars = i->second.vars;
2752
2753 // Parse the targets and the variable assignments.
2754 // Mark the targets as dependencies of the job targets.
2755 dependency_t &dep = *dependencies[i->second.rule.targets.front()];
2757 char const *p = &buf[0] + sizeof(int);
2758 while (true)
2759 {
2760 len = strlen(p);
2761 if (len == 0)
2762 {
2763 ++waiting_jobs;
2764 break;
2765 }
2766 switch (*p)
2767 {
2768 case 'T':
2769 {
2770 if (len == 1) goto error;
2771 std::string target(p + 1, p + len);
2772 DEBUG << "adding dependency " << target << " to job\n";
2773 proc->pending.push_back(target);
2774 dep.deps.insert(target);
2775 break;
2776 }
2777 case 'V':
2778 {
2779 if (len == 1) goto error;
2780 std::string var(p + 1, p + len);
2781 DEBUG << "adding variable " << var << " to job\n";
2782 last_var = &proc->vars[var];
2783 last_var->clear();
2784 break;
2785 }
2786 case 'W':
2787 {
2788 if (!last_var) goto error;
2789 last_var->push_back(std::string(p + 1, p + len));
2790 break;
2791 }
2792 default:
2793 goto error;
2794 }
2795 p += len + 1;
2796 }
2797
2798 if (!propagate_vars && !proc->vars.empty())
2799 {
2800 std::cerr << "Assignments are ignored unless 'variable-propagation' is enabled" << std::endl;
2801 proc->vars.clear();
2802 }
2803}
2804
2805/**
2806 * Handle child process exit status.
2807 */
2808static void finalize_job(pid_t pid, bool res)
2809{
2810 pid_job_map::iterator i = job_pids.find(pid);
2811 assert(i != job_pids.end());
2812 int job_id = i->second;
2813 job_pids.erase(i);
2814 --running_jobs;
2815 complete_job(job_id, res);
2816}
2817
2818/**
2819 * Loop until all the jobs have finished.
2820 *
2821 * @post There are no client requests left, not even virtual ones.
2822 */
2823static void server_loop()
2824{
2825 while (handle_clients())
2826 {
2827 DEBUG_open << "Handling events... ";
2828 #ifdef WINDOWS
2829 size_t len = job_pids.size() + 1;
2830 HANDLE h[len];
2831 int num = 0;
2832 for (pid_job_map::const_iterator i = job_pids.begin(),
2833 i_end = job_pids.end(); i != i_end; ++i, ++num)
2834 {
2835 h[num] = i->first;
2836 }
2838 h[num] = aev;
2843 if (len <= w)
2844 continue;
2845 if (w == len - 1)
2846 {
2847 accept_client();
2848 continue;
2849 }
2850 pid_t pid = h[w];
2851 DWORD s = 0;
2852 bool res = GetExitCodeProcess(pid, &s) && s == 0;
2855 #else
2858 fd_set fdset;
2859 FD_ZERO(&fdset);
2861 int ret = pselect(socket_fd + 1, &fdset, NULL, NULL, NULL, &emptymask);
2862 if (ret > 0 /* && FD_ISSET(socket_fd, &fdset)*/) accept_client();
2863 if (!got_SIGCHLD) continue;
2864 got_SIGCHLD = 0;
2865 pid_t pid;
2866 int status;
2867 while ((pid = waitpid(-1, &status, WNOHANG)) > 0)
2868 {
2869 bool res = WIFEXITED(status) && WEXITSTATUS(status) == 0;
2871 }
2872 #endif
2873 }
2874
2875 assert(clients.empty());
2876}
2877
2878/**
2879 * Load dependencies and rules, listen to client requests, and loop until
2880 * all the requests have completed.
2881 * If Remakefile is obsolete, perform a first run with it only, then reload
2882 * the rules, and perform a second with the original clients.
2883 */
2884static void server_mode(std::string const &remakefile, string_list const &targets)
2885{
2889 create_server();
2891 {
2892 clients.push_back(client_t());
2893 clients.back().pending.push_back(remakefile);
2894 server_loop();
2895 if (build_failure) goto early_exit;
2897 specific_rules.clear();
2898 generic_rules.clear();
2899 first_target.clear();
2901 }
2902 clients.push_back(client_t());
2903 if (!targets.empty()) clients.back().pending = targets;
2904 else if (!first_target.empty())
2905 clients.back().pending.push_back(first_target);
2906 server_loop();
2907 early_exit:
2909#ifndef WINDOWS
2912#endif
2915 {
2916 std::cout << "remake: Leaving directory `" << prefix_dir << '\'' << std::endl;
2917 }
2919}
2920
2921/** @} */
2922
2923/**
2924 * @defgroup client Client
2925 *
2926 * @{
2927 */
2928
2929/**
2930 * Connect to the server @a socket_name, send a request for building @a targets
2931 * with some @a variables, and exit with the status returned by the server.
2932 */
2933static void client_mode(char *socket_name, string_list const &targets)
2934{
2935 if (false)
2936 {
2937 error:
2938 perror("Failed to send targets to server");
2940 }
2941 if (targets.empty()) exit(EXIT_SUCCESS);
2942 DEBUG_open << "Connecting to server... ";
2943
2944 // Connect to server.
2945#ifdef WINDOWS
2946 struct sockaddr_in socket_addr;
2947 socket_fd = socket(AF_INET, SOCK_STREAM, 0);
2948 if (socket_fd == INVALID_SOCKET) goto error;
2949 socket_addr.sin_family = AF_INET;
2950 socket_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
2951 socket_addr.sin_port = atoi(socket_name);
2952 if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(sockaddr_in)))
2953 goto error;
2954#else
2955 struct sockaddr_un socket_addr;
2956 size_t len = strlen(socket_name);
2957 if (len >= sizeof(socket_addr.sun_path) - 1) exit(EXIT_FAILURE);
2958 socket_fd = socket(AF_UNIX, SOCK_STREAM, 0);
2959 if (socket_fd == INVALID_SOCKET) goto error;
2960 socket_addr.sun_family = AF_UNIX;
2961 strcpy(socket_addr.sun_path, socket_name);
2962 if (connect(socket_fd, (struct sockaddr *)&socket_addr, sizeof(socket_addr.sun_family) + len))
2963 goto error;
2964#ifdef MACOSX
2965 int set_option = 1;
2967 goto error;
2968#endif
2969#endif
2970
2971 // Send current job id.
2972 char *id = getenv("REMAKE_JOB_ID");
2973 int job_id = id ? atoi(id) : -1;
2974 if (send(socket_fd, (char *)&job_id, sizeof(job_id), MSG_NOSIGNAL) != sizeof(job_id))
2975 goto error;
2976
2977 // Send targets.
2978 for (string_list::const_iterator i = targets.begin(),
2979 i_end = targets.end(); i != i_end; ++i)
2980 {
2981 DEBUG_open << "Sending target " << *i << "... ";
2982 std::string s = 'T' + *i;
2983 ssize_t len = s.length() + 1;
2984 if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2985 goto error;
2986 }
2987
2988 // Send variables.
2989 for (variable_map::const_iterator i = variables.begin(),
2990 i_end = variables.end(); i != i_end; ++i)
2991 {
2992 DEBUG_open << "Sending variable " << i->first << "... ";
2993 std::string s = 'V' + i->first;
2994 ssize_t len = s.length() + 1;
2995 if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
2996 goto error;
2997 for (string_list::const_iterator j = i->second.begin(),
2998 j_end = i->second.end(); j != j_end; ++j)
2999 {
3000 std::string s = 'W' + *j;
3001 len = s.length() + 1;
3002 if (send(socket_fd, s.c_str(), len, MSG_NOSIGNAL) != len)
3003 goto error;
3004 }
3005 }
3006
3007 // Send terminating nul and wait for reply.
3008 char result = 0;
3009 if (send(socket_fd, &result, 1, MSG_NOSIGNAL) != 1) goto error;
3010 if (recv(socket_fd, &result, 1, 0) != 1) exit(EXIT_FAILURE);
3012}
3013
3014/** @} */
3015
3016/**
3017 * @defgroup ui User interface
3018 *
3019 * @{
3020 */
3021
3022/**
3023 * Display usage and exit with @a exit_status.
3024 */
3025static void usage(int exit_status)
3026{
3027 std::cerr << "Usage: remake [options] [target] ...\n"
3028 "Options\n"
3029 " -B, --always-make Unconditionally make all targets.\n"
3030 " -d Echo script commands.\n"
3031 " -d -d Print lots of debugging information.\n"
3032 " -f FILE Read FILE as Remakefile.\n"
3033 " -h, --help Print this message and exit.\n"
3034 " -j[N], --jobs=[N] Allow N jobs at once; infinite jobs with no arg.\n"
3035 " -k, --keep-going Keep going when some targets cannot be made.\n"
3036 " -r Look up targets from the dependencies on stdin.\n"
3037 " -s, --silent, --quiet Do not echo targets.\n";
3039}
3040
3041/**
3042 * This program behaves in two different ways.
3043 *
3044 * - If the environment contains the REMAKE_SOCKET variable, the client
3045 * connects to this socket and sends to the server its build targets.
3046 * It exits once it receives the server reply.
3047 *
3048 * - Otherwise, it creates a server that waits for build requests. It
3049 * also creates a pseudo-client that requests the targets passed on the
3050 * command line.
3051 */
3052int main(int argc, char *argv[])
3053{
3054 std::string remakefile;
3055 string_list targets;
3056 bool literal_targets = false;
3057 bool indirect_targets = false;
3058
3059 // Parse command-line arguments.
3060 for (int i = 1; i < argc; ++i)
3061 {
3062 std::string arg = argv[i];
3063 if (arg.empty()) usage(EXIT_FAILURE);
3064 if (literal_targets) goto new_target;
3065 if (arg == "-h" || arg == "--help") usage(EXIT_SUCCESS);
3066 if (arg == "-d")
3067 if (echo_scripts) debug.active = true;
3068 else echo_scripts = true;
3069 else if (arg == "-k" || arg =="--keep-going")
3070 keep_going = true;
3071 else if (arg == "-s" || arg == "--silent" || arg == "--quiet")
3072 show_targets = false;
3073 else if (arg == "-r")
3074 indirect_targets = true;
3075 else if (arg == "-B" || arg == "--always-make")
3076 obsolete_targets = true;
3077 else if (arg == "-f")
3078 {
3079 if (++i == argc) usage(EXIT_FAILURE);
3080 remakefile = argv[i];
3081 }
3082 else if (arg == "--")
3083 literal_targets = true;
3084 else if (arg.compare(0, 2, "-j") == 0)
3085 max_active_jobs = atoi(arg.c_str() + 2);
3086 else if (arg.compare(0, 7, "--jobs=") == 0)
3087 max_active_jobs = atoi(arg.c_str() + 7);
3088 else
3089 {
3090 if (arg[0] == '-') usage(EXIT_FAILURE);
3091 if (arg.find('=') != std::string::npos)
3092 {
3093 std::istringstream in(arg);
3094 std::string name = read_word(in);
3095 if (name.empty() || !expect_token(in, Equal)) usage(EXIT_FAILURE);
3096 read_words(in, variables[name]);
3097 continue;
3098 }
3099 new_target:
3100 targets.push_back(arg);
3101 DEBUG << "New target: " << arg << '\n';
3102 }
3103 }
3104
3107
3108 if (indirect_targets)
3109 {
3110 load_dependencies(std::cin);
3111 string_list l;
3112 targets.swap(l);
3113 if (l.empty() && !dependencies.empty())
3114 {
3115 l.push_back(dependencies.begin()->second->targets.front());
3116 }
3117 for (string_list::const_iterator i = l.begin(),
3118 i_end = l.end(); i != i_end; ++i)
3119 {
3120 dependency_map::const_iterator j = dependencies.find(*i);
3121 if (j == dependencies.end()) continue;
3122 dependency_t const &dep = *j->second;
3123 for (string_set::const_iterator k = dep.deps.begin(),
3124 k_end = dep.deps.end(); k != k_end; ++k)
3125 {
3126 targets.push_back(normalize(*k, working_dir, working_dir));
3127 }
3128 }
3129 dependencies.clear();
3130 }
3131
3132#ifdef WINDOWS
3134 if (WSAStartup(MAKEWORD(2,2), &wsaData))
3135 {
3136 std::cerr << "Unexpected failure while initializing Windows Socket" << std::endl;
3137 return 1;
3138 }
3139#endif
3140
3141 // Run as client if REMAKE_SOCKET is present in the environment.
3142 if (char *sn = getenv("REMAKE_SOCKET")) client_mode(sn, targets);
3143
3144 // Otherwise run as server.
3145 if (remakefile.empty())
3146 {
3147 remakefile = "Remakefile";
3149 }
3151 server_mode(remakefile, targets);
3152}
3153
3154/** @} */
static void client_mode(char *socket_name, string_list const &targets)
Definition remake.cpp:2933
static void save_dependencies()
Definition remake.cpp:1487
static void load_dependencies()
Definition remake.cpp:1471
static bool skip_eol(std::istream &in, bool multi=false)
Definition remake.cpp:1045
static int expect_token(std::istream &in, int mask)
Definition remake.cpp:1074
static void skip_empty(std::istream &in)
Definition remake.cpp:1034
static std::string read_word(std::istream &in, bool detect_equal=true)
Definition remake.cpp:1120
static void skip_spaces(std::istream &in)
Definition remake.cpp:1024
@ Plusequal
Definition remake.cpp:1064
@ Word
Definition remake.cpp:1058
@ Equal
Definition remake.cpp:1060
@ Unexpected
Definition remake.cpp:1057
@ Colon
Definition remake.cpp:1059
@ Dollarpar
Definition remake.cpp:1061
@ Pipe
Definition remake.cpp:1065
@ Comma
Definition remake.cpp:1063
@ Rightpar
Definition remake.cpp:1062
static void load_rules(std::string const &remakefile)
Definition remake.cpp:1775
static void register_transparent_rule(rule_t const &rule, string_list const &targets)
Definition remake.cpp:1529
static void register_scripted_rule(rule_t const &rule)
Definition remake.cpp:1572
static void register_rule(rule_t const &rule)
Definition remake.cpp:1601
static void load_rule(std::istream &in, std::string const &first)
Definition remake.cpp:1626
static std::string normalize_abs(std::string const &s, std::string const &p)
Definition remake.cpp:931
static std::string normalize(std::string const &s, std::string const &w, std::string const &p)
Definition remake.cpp:953
static void init_working_dir()
Definition remake.cpp:875
static void init_prefix_dir()
Definition remake.cpp:897
static void normalize_list(string_list &l, std::string const &w, std::string const &p)
Definition remake.cpp:1004
static void substitute_pattern(std::string const &pat, string_list const &src, string_list &dst)
Definition remake.cpp:1869
static void instantiate_rule(std::string const &target, rule_t const &src, rule_t &dst)
Definition remake.cpp:1885
static void find_generic_rule(job_t &job, std::string const &target)
Definition remake.cpp:1916
static void merge_rule(rule_t &dest, rule_t const &src)
Definition remake.cpp:1846
static void find_rule(job_t &job, std::string const &target)
Definition remake.cpp:1930
static void complete_request(client_t &client, bool success)
Definition remake.cpp:2436
static void accept_client()
Definition remake.cpp:2689
static bool handle_clients()
Definition remake.cpp:2493
static void create_server()
Definition remake.cpp:2609
static void finalize_job(pid_t pid, bool res)
Definition remake.cpp:2808
static std::string prepare_script(job_t const &job)
Definition remake.cpp:2173
static status_e start(std::string const &target, client_list::iterator &current)
Definition remake.cpp:2378
static status_e run_script(int job_id, job_t const &job)
Definition remake.cpp:2256
static bool has_free_slots()
Definition remake.cpp:2470
static void complete_job(int job_id, bool success, bool started=true)
Definition remake.cpp:2131
static void server_loop()
Definition remake.cpp:2823
static void server_mode(std::string const &remakefile, string_list const &targets)
Definition remake.cpp:2884
static bool still_need_rebuild(std::string const &target)
Definition remake.cpp:2097
static void update_status(std::string const &target)
Definition remake.cpp:2064
static status_t const & get_status(std::string const &target)
Definition remake.cpp:1987
input_status next(std::string &)
Definition remake.cpp:1218
addprefix_generator(input_generator const &, bool &)
Definition remake.cpp:1318
variable_generator(std::string const &, variable_map const *)
Definition remake.cpp:1199
static bool read_words(input_generator &in, string_list &res)
Definition remake.cpp:1286
input_status next(std::string &)
Definition remake.cpp:1384
input_status next(std::string &)
Definition remake.cpp:1246
addsuffix_generator(input_generator const &, bool &)
Definition remake.cpp:1374
static generator * get_function(input_generator const &, std::string const &)
Definition remake.cpp:1414
input_status
Definition remake.cpp:1173
input_status next(std::string &)
Definition remake.cpp:1328
@ SyntaxError
Definition remake.cpp:1175
@ Eof
Definition remake.cpp:1176
@ Success
Definition remake.cpp:1174
int main(int argc, char *argv[])
Definition remake.cpp:3052
static void usage(int exit_status)
Definition remake.cpp:3025
static bool keep_going
Definition remake.cpp:665
@ INVALID_SOCKET
Definition remake.cpp:461
static int max_active_jobs
Definition remake.cpp:659
static int job_counter
Definition remake.cpp:692
static bool build_failure
Definition remake.cpp:702
static struct log debug
Definition remake.cpp:801
static void sigchld_handler(int)
Definition remake.cpp:761
std::map< int, job_t > job_map
Definition remake.cpp:582
std::map< std::string, status_t > status_map
Definition remake.cpp:542
int socket_t
Definition remake.cpp:460
std::map< std::string, ref_ptr< rule_t > > rule_map
Definition remake.cpp:570
static client_list clients
Definition remake.cpp:653
static int waiting_jobs
Definition remake.cpp:686
static time_t now
Definition remake.cpp:729
static std::string first_target
Definition remake.cpp:714
static rule_list generic_rules
Definition remake.cpp:632
std::list< std::string > string_list
Definition remake.cpp:469
static std::string working_dir
Definition remake.cpp:734
static status_map status
Definition remake.cpp:627
std::list< rule_t > rule_list
Definition remake.cpp:568
#define DEBUG_close
Definition remake.cpp:817
status_e
Definition remake.cpp:523
@ Failed
Build failed for target.
Definition remake.cpp:530
@ Todo
Target is missing or obsolete.
Definition remake.cpp:525
@ Running
Target is being rebuilt.
Definition remake.cpp:527
@ Recheck
Target has an obsolete dependency.
Definition remake.cpp:526
@ Remade
Target was successfully rebuilt.
Definition remake.cpp:529
@ Uptodate
Target is up-to-date.
Definition remake.cpp:524
@ RunningRecheck
Static prerequisites are being rebuilt.
Definition remake.cpp:528
static dependency_map dependencies
Definition remake.cpp:622
static std::ostream & operator<<(std::ostream &out, escape_string const &se)
Definition remake.cpp:834
static variable_map variables
Definition remake.cpp:617
std::set< std::string > string_set
Definition remake.cpp:471
static bool obsolete_targets
Definition remake.cpp:754
static bool changed_prefix_dir
Definition remake.cpp:744
static job_map jobs
Definition remake.cpp:642
std::map< pid_t, int > pid_job_map
Definition remake.cpp:584
static char * socket_name
Definition remake.cpp:708
static bool show_targets
Definition remake.cpp:719
static pid_job_map job_pids
Definition remake.cpp:647
char ** environ
std::map< std::string, ref_ptr< dependency_t > > dependency_map
Definition remake.cpp:515
std::map< std::string, assign_t > assign_map
Definition remake.cpp:553
static bool echo_scripts
Definition remake.cpp:724
static int running_jobs
Definition remake.cpp:678
static sigset_t old_sigmask
Definition remake.cpp:757
std::map< std::string, string_list > variable_map
Definition remake.cpp:517
#define DEBUG_open
Definition remake.cpp:816
#define DEBUG
Definition remake.cpp:815
static rule_map specific_rules
Definition remake.cpp:637
static socket_t socket_fd
Definition remake.cpp:697
static std::string prefix_dir
Definition remake.cpp:739
static bool propagate_vars
Definition remake.cpp:749
static void sigint_handler(int)
Definition remake.cpp:766
static volatile sig_atomic_t got_SIGCHLD
Definition remake.cpp:759
std::list< client_t > client_list
Definition remake.cpp:611
input_generator gen
Definition remake.cpp:1309
string_list::const_iterator prei
Definition remake.cpp:1311
std::string suf
Definition remake.cpp:1313
string_list pre
Definition remake.cpp:1310
string_list suf
Definition remake.cpp:1366
string_list::const_iterator sufi
Definition remake.cpp:1367
input_generator gen
Definition remake.cpp:1365
std::string pre
Definition remake.cpp:1369
bool append
Definition remake.cpp:549
string_list value
Definition remake.cpp:550
string_list pending
Targets not yet started.
Definition remake.cpp:604
socket_t socket
Socket used to reply to the client (invalid for pseudo clients).
Definition remake.cpp:601
bool delayed
Whether it is a dependency client and a script has to be started on request completion.
Definition remake.cpp:607
bool failed
Whether some targets failed in mode -k.
Definition remake.cpp:603
int job_id
Job for which the built script called remake and spawned the client (negative for original clients).
Definition remake.cpp:602
string_set running
Targets being built.
Definition remake.cpp:605
variable_map vars
Variables set on request.
Definition remake.cpp:606
string_list targets
Definition remake.cpp:511
string_set deps
Definition remake.cpp:512
escape_string(std::string const &s)
Definition remake.cpp:827
std::string const & input
Definition remake.cpp:826
virtual ~generator()
Definition remake.cpp:1184
virtual input_status next(std::string &)=0
std::istream & in
Definition remake.cpp:1234
generator * nested
Definition remake.cpp:1235
variable_map const * local_variables
Definition remake.cpp:1236
input_generator(std::istream &i, variable_map const *lv, bool e=false)
Definition remake.cpp:1238
variable_map vars
Values of local variables.
Definition remake.cpp:579
rule_t rule
Original rule.
Definition remake.cpp:578
bool open
Definition remake.cpp:776
std::ostream & operator()(bool o)
Definition remake.cpp:789
bool active
Definition remake.cpp:776
int depth
Definition remake.cpp:777
log()
Definition remake.cpp:778
std::ostream & operator()()
Definition remake.cpp:781
content(T const &t)
Definition remake.cpp:486
~ref_ptr()
Definition remake.cpp:492
ref_ptr(T const &t)
Definition remake.cpp:490
T & operator*() const
Definition remake.cpp:501
ref_ptr()
Definition remake.cpp:489
ref_ptr(ref_ptr const &p)
Definition remake.cpp:491
content * ptr
Definition remake.cpp:488
T * operator->() const
Definition remake.cpp:506
ref_ptr & operator=(ref_ptr const &p)
Definition remake.cpp:493
assign_map assigns
Assignment of variables.
Definition remake.cpp:563
string_list wdeps
Like deps, except that they are not registered as dependencies.
Definition remake.cpp:562
std::string script
Shell script for building the targets.
Definition remake.cpp:565
string_list targets
Files produced by this rule.
Definition remake.cpp:560
std::string stem
Stem used to instantiate the rule, if any.
Definition remake.cpp:564
string_list deps
Dependencies used for an implicit call to remake at the start of the script.
Definition remake.cpp:561
status_e status
Actual status.
Definition remake.cpp:538
time_t last
Last-modified date.
Definition remake.cpp:539
string_list::const_iterator vend
Definition remake.cpp:1194
string_list::const_iterator vcur
Definition remake.cpp:1194
std::string name
Definition remake.cpp:1193