Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions shadowdash_docs/new_langauge_doc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# ShadowDash New Language Design Manifest Documentation

The `shadowdash_test/manifest.h` file defines the interface and internal logic for ShadowDash, a build configuration framework inspired by Ninja. This document is structured to first introduce **how to use the framework** and then dive into the **design logic**.

---

## How to Use ShadowDash with Newly Designed Language

The primary user-facing elements of ShadowDash are macros designed for defining rules, builds, pools, and defaults concisely. Below are the macros and how to use them, followed by a full example.

### 1. Key Macros

| Macro | Description | Example |
|-----------------|-----------------------------------------------------|-------------------------------------------------------------------------|
| `let` | Define a variable with a name and value. | `let(my_var, "value"_l);` |
| `bind` | Create a key-value binding for a rule or build. | `bind(command, "g++ -c in"_v);` |
| `new_pool` | Define a resource pool with a maximum concurrency. | `new_pool(my_pool, 4);` |
| `default` | Set default build targets. | `default("all"_l, "test"_l);` |
| `list` | Define a list of strings or tokens. | `list("item1"_l, "item2"_l);` |
| `rule` | Define a rule with specific bindings. | `rule(compile, bind(command, "g++ -c in"_v, "-o out"_v));` |
| `str` | Create a string composed of literals or variables. | `str("hello.o"_l);` |
| `ShadowDash` | Finalize the manifest with all builds and settings. | `ShadowDash({build1, build2}, {pool1}, default_targets);` |

### 2. Full Example

shadowdash_test/manifest_example.cc is a complete example of using the macros in `manifest.h` to define a build configuration.

---

## New Language Design Logic

### 1. Internal Components

The new language design revolves around the following key classes:

#### Token
Represents a basic element in the build system, with types:
- **LITERAL**: A fixed string value.
- **VAR**: A variable that can be dynamically substituted.

#### String Abstraction (`str_`)
Allows combining `Token`s into complex strings, useful for commands and paths.

#### Lists and Bindings
- **`list_`**: A collection of `str_` instances.
- **`binding`**: Key-value pairs used in rules to define behavior.

#### Rules
The `rule__` class defines how to process inputs, with:
- **Regular rules**: Contain bindings for commands.
- **Special rules**: Predefined behaviors like `phony`.

#### Builds
The `build` class represents a single build operation, with inputs, outputs, and an associated rule.

#### Pools
The `pool` class defines resource limits for concurrent tasks.

#### Defaults
The `shadowdash_default` class sets the default targets.

#### ShadowDash Manifest
The `ShadowDash_` class aggregates all builds, pools, and default targets into a single manifest.

---

## Summary

The `shadowdash_test/manifest.h` file provides:
1. **User-friendly macros** for defining builds and rules.
2. **Robust internal logic** to manage complex dependencies.
3. **Extensibility** for diverse project requirements.

By focusing on the foundational design of ShadowDash’s new language, this module ensures seamless integration with other teams (Converter and Hello World Graph).
For further details, refer to the example and test cases provided in the project repository.
1 change: 1 addition & 0 deletions shadowdash_test/manifest.h
58 changes: 58 additions & 0 deletions shadowdash_test/manifest_example.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#include "manifest.h"

manifest() {
// Define a "compile" rule that complies into object files.
rule(
compile,
bind(command, {"g++", "-c", "in"_v, "-o", "out"_v, "-MD", "-MF", "depfile"_v}),
bind(depfile, {"out"_v, ".d"}),
bind(deps, {"gcc"}),
);

// Define a "link" rule that links object files into an executable.
rule(
link,
bind(command, {"g++", "in"_v, "-o", "out"_v}),
);

// Define a build configuration for compiling "hello.c" into object files.
auto build1 = build(list(str("hello.o"), str("hello2.o")),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For build1, build2 and build3, these names are generic and do not convey the purpose of the builds. Consider renaming them to reflect the specific target, like buildHelloObjects.

list(),
compile,
list(str("hello.c")),
list(),
list(),
{ bind(test, {"123"}) }
);

// Define a build configuration for linking "hello.o" into an executable.
auto build2 = build(list(str("hello")),
list(),
link,
list(str("hello.o")),
list(),
list(),
{ }
);

// Define a build configuration with the special rule "phony"
auto build3 = build(list(str("hello.d")),
list(),
phony,
list(),
list(),
list(),
{ }
);

// Define pools with a specified concurrency depth limit.
new_pool(pool1, 1);
new_pool(pool2, 1);
new_pool(pool3, 1);

// Define default targets to be built when no specific target is specified.
default(str("all"), str("test"));

// Create the ShadowDash manifest with all builds, pools, and default targets defined above.
ShadowDash({ build1, build2, build3 }, {pool1, pool2, pool3}, default_target);
}
195 changes: 195 additions & 0 deletions src/shadowdash_manifest.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
#pragma once

#include <iostream>
#include <string>
#include <vector>
#include <utility>
#include <optional>

namespace shadowdash {

class Token {
public:
enum Type { LITERAL, VAR };

Token(Type type, std::string value)
: type_(type), value_(value) {}

Token(std::string value) : Token(Token::LITERAL, value) {}

Token(const char* value) : Token(Token::LITERAL, std::string(value)) {}

Type type_;
std::string value_;
};

Token operator"" _l(const char* value, std::size_t len) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For operator"" _l and operator"" _v, while these are clever, the naming might not be immediately clear to someone unfamiliar with the codebase. Consider adding inline comments explaining what these operators do, or using more expressive suffixes than _l and _v.

return Token(Token::Type::LITERAL, std::string(value, len));
}

Token operator"" _v(const char* value, std::size_t len) {
return Token(Token::Type::VAR, std::string(value, len));
}

class str_ {
public:
str_(std::vector<Token> tokens) : tokens_(tokens) {}

std::vector<Token> tokens_;
};

using binding = std::pair<std::string, str_>;
using map = std::vector<binding>;

class list_ {
public:
list_(std::vector<str_> values) : values_(values) {}

std::vector<str_> values_;
};

class var {
public:
var(const char* name, str_ value) : name_(name), value_(value) {}

const char* name_;
str_ value_;
};

class rule__ {
public:
enum SPECIAL_RULE {
phony
};

union _rule {
_rule() : sp_rule(phony) {}
_rule(map bindings) : bindings_(bindings) {}
_rule(SPECIAL_RULE sr) : sp_rule(sr) {}
~_rule() {}

map bindings_;
SPECIAL_RULE sp_rule;
} _rule_data;
bool is_special;

// Constructor for regular and special rules
rule__(map bindings) : _rule_data(bindings), is_special(false) {}
rule__(SPECIAL_RULE sp_rule) : _rule_data(sp_rule), is_special(true) {}

// Copy constructor, handles both regular and special rules
rule__(const rule__& other) : is_special(other.is_special) {
if (is_special) {
new (&_rule_data) _rule(other._rule_data.sp_rule);
} else {
new (&_rule_data) _rule(other._rule_data.bindings_);
}
}
};
#define phony shadowdash::rule__::phony

class build {
public:
build(
list_ outputs,
list_ implicit_outputs,
rule__ rule,
list_ inputs,
list_ implicit_inputs,
list_ order_only_inputs,
map bindings)
: outputs_(outputs),
implicit_outputs_(implicit_outputs),
rule_(rule),
inputs_(inputs),
implicit_inputs_(implicit_inputs),
order_only_inputs_(order_only_inputs),
bindings_(bindings) {
}

list_ outputs_;
list_ implicit_outputs_;
rule__ rule_;
list_ inputs_;
list_ implicit_inputs_;
list_ order_only_inputs_;
map bindings_;
std::string pool;
};

// Define a resource pool with name and maximum depth
class pool {
public:
pool(
str_ name,
int depth)
: name_(name),
depth_(depth) {}; // Maximum allowable concurrent tasks in this pool.

str_ name_;
int depth_;
};

// Represent a default target in the build configuration
class shadowdash_default {
public:
shadowdash_default(
list_ target)
: target_(target) {};

list_ target_;
};

class ShadowDash_ {
public:
ShadowDash_(std::vector<build> builds) : builds(builds), pools({}), default_({}) {}
ShadowDash_(std::vector<build> builds, std::vector<pool> pools) : builds(builds), pools(pools), default_({}) {}
ShadowDash_(std::vector<build> builds, std::vector<pool> pools, shadowdash_default default_) : builds(builds), pools(pools), default_({default_}) {}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is too long, should split this line into two.


std::vector<build> builds;
std::vector<pool> pools;
std::vector<shadowdash_default> default_;
};

static auto in = "in"_v;
static auto out = "out"_v;

} // namespace shadowdash

// Macors to simplify the syntax for defining vairables, offering a concise way to create ShadowDash configs.
#define let(name, ...) \
var name { \
#name, str_ { \
__VA_ARGS__ \
} \
}

#define bind(name, ...) \
{ \
#name, str_ { \
__VA_ARGS__ \
} \
}

#define new_pool(name, depth) \
pool name = pool(str({#name}), depth)

#define default(...) \
auto default_target = shadowdash_default(list(__VA_ARGS__));

#define list(...) \
list_{{__VA_ARGS__}}

#define rule(name, ...) \
rule__ name = rule__{{__VA_ARGS__}}

#define str(...) \
str_{{__VA_ARGS__}}

#define ShadowDash return ShadowDash_

#define manifest ShadowDash_ manifest

#ifndef SHADOWDASH_SOURCE
using namespace shadowdash;
#endif