A next-generation package manager for the front-end

Suppose I'm using an <img> tag in a .html template, duo would never be made aware of such an asset. (and thus it would never be moved to build/) In component, this is where manifests came in handy, so I'm wondering how duo will work with such a case?

Will it just be up to the developer to have a script that copies known images to build/ after duo has run? Is there any way duo could be notified of these assets? (when does duo look at manifest files?)

I think this will allow us to continue to support components with JS and CSS. It's a little weird though, because there should probably only be one style that imports everything else.

@yields @ianstormtaylor @visionmedia could use some thoughts here.

@MatthewMueller this is all it would take. 5 tests don't pass with this though so it will probably expand some. but, this solves the problem of being able to support both legacy component and the new @import style, so you can port stuff from component to duo without breaking anything in the process.

At a high level, basically what happens is, if you specify a specific version in your component.json for some component, and some dependencies also reference that component, but they all have * instead of a specific version, then all the nested components will use the latest version of that component, while anything in your current project will use the specific version.

What should happen is, if all dependencies reference some component using * version, and then in your current project you specify 0.3.1, then everything should use 0.3.1, even though the latest may be 0.7.0 or something.

In more depth, here's the setup.

current project:

{
  "name": "my-app",
  "dependencies": {
    "some/component": "0.3.1",
    "dependency/a": "*",
    "dependency/b": "1.2.3"
  }
}

the component you want to pin to a specific version:

{
  "repo": "some/component",
  "version": "0.7.0"
}

some random dependency that depends on some/component, but isn't tied to a specific version:

{
  "repo": "dependency/a",
  "version": "2.4.1",
  "dependencies": {
    "some/component": "*"
  }
}

some other random dependency that depends on some/component, but isn't tied to a specific version either:

{
  "repo": "dependency/b",
  "version": "1.2.3",
  "dependencies": {
    "some/component": "*"
  }
}

The problem now is this. Say inside dependency/a or dependency/b, you do this:

// say this is dependency-a/index.js
require('some/component');

Now say in your current project you do this:

require('dependency/a');

When you require dependency/a in your current project, which then requires some/component, it will require some/[email protected]. This isn't the desired behavior I don't think (this isn't how it works in node).

Instead, it should require some/component/@0.3.1, the one that is defined in the current project's component.json.

Otherwise, you have no control over what the nested components might download. The * that they have their versions pinned at means that you can pin your current project to a specific version and they will use that (in my mind at least, and this is how node/component did it).

Seems like a bug to me, what do you think? Any ideas how to fix it?

I am getting this issue when trying to build a project being migrated from component.js:

⧑ duo --no-cache --use duo-env lib/index.{js,css}

        using : duo-env
     building : lib/index.js
     building : lib/index.css
    installed : [email protected]
    ...
    installed : component-type@c817c
        error : Error: EEXIST, symlink '/Users/jasonkuhrt/projects/cloud-app-cloud-control/components/[email protected]/lib/number-readout/assets/digital-7_mono_italic-webfont.eot'

It seems I am the first to have an EEXIST issue on duo; any ideas anyone?

This adds support for external source-maps!

The --development flag no longer implies source-maps, instead the --source-maps (or --inline-source-maps) flags must be used.

Duo#sourceMap(value) has been added to the API.

• when value === true, an external source-map will be created. (ex: build/index.js
  • build/index.js.map)
• when value === 'inline', the output file will have the sourcemap added as an inline comment.
• when value === false, no source-maps will be generated at all.

In the CLI, when printing to stdout, only inline source-maps are supported. Thus, if you only specify true, it will automatically change to inline.

BREAKING: this changes the api for Duo#run(), it now returns the same object that duo-pack@2 returns. (ie: { code, map })

i've been thinking a lot about file transforms lately and I think I've come up with a decent solution. Here's what the API would look like:

duo
  .use(jade())
  .use(styl())

Where styl and jade are one of the following:

• a synchronous function that returns a string function(...) { return compiled }
• an asynchronous function: function(..., fn) { return fn(null, compiled) }
• a gulp stream

The reason I'm pushing for gulp streams is that it will allow us to tap into a huge repository of plugins for things like: transpiling ES6 => ES5 (adding import, etc), coffeescript support, and many other things I haven't thought of.

However, the problem I'm running into is the following. Maybe this is obvious to someone that's worked a lot with streams, but basically if you do the following, you will not get the intended results:

var fs = require('fs');
var map = require('map-stream');

function prepend(str, fn) {
  return fn(null, 'hi ' + str);
};

var t = map(prepend);

var a = fs.createReadStream('./a.txt', 'utf8');
var b = fs.createReadStream('./b.txt', 'utf8');
var c = fs.createReadStream('./c.txt', 'utf8');

a.pipe(t).pipe(process.stdout);
b.pipe(t).pipe(process.stdout);
c.pipe(t).pipe(process.stdout);

Basically, the transform stream pulls from all 3 sources at the same time. This is a problem because gulp streams are already initialized by the time they're passed through (ie. styl()).

What needs to happen is the files need to be queued and wait until the transform pipeline is free. I'm not sure if this is going to be a bottleneck yet, or if there is a better way, but that's what I'm facing right now.

I guess to get duo out there maybe we can just support the first 2 options (sync and async fn), but I think the gulp streams will be a huge plus for people.

How we'll walk the files is the following:

$ duo main.css

main.css:

@import "./one.styl"
@import "./two.less"
@import "./three"

To resolve three we will base it off the root entry (extname("main.css") => ".css" => "./three.css"). We will do the same for JS and eventually HTML.

/cc @yields @juliangruber @ianstormtaylor

Duo currently ignores the "local" and "paths" entries in component.json when resolving dependencies. This change resolves that by adding an isLocal test when resolving. It appends the dependency to all of the paths listed in the "paths" entry (and also tries root), returning the first one for which fs.exists reports true.

The isLocal test only gets used when component.json includes a "locals" entry. Otherwise, dependency resolution remains unchanged.

Really only speculating still, but this should fix #221.

See:

• https://github.com/duojs/duo/pull/118#discussion_r15720060
• https://github.com/duojs/duo/issues/221

I've installed duo (globally) without problems, created a token and a .netrc file (which is actually a _netrc file with windows: http://stackoverflow.com/q/6031214).

But after trying the "getting started" example from the readme, I get the following error (when running duo index.js > build.js):

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: spawn ENOENT
    at errnoException (child_process.js:988:11)
    at Process.ChildProcess._handle.onexit (child_process.js:779:34)

An empty build.js file is created and duo exits after the error. Any idea what's going wrong here?

Btw, these are the contents of the index.js I'm trying to build:

var uid = require('matthewmueller/uid');
var fmt = require('yields/fmt');

var msg = fmt('Your unique ID is %s!', uid());
window.alert(msg);

Hmm, not totally sure what this could be just running into it. I haven't changed anything since yesterday but now when I do a fresh checkout of https://github.com/segmentio/analytics.js-integrations, and build, it's giving me this:

$ duo --development test/index.js build/build.js
/usr/local/lib/node_modules/duo/node_modules/duo-package/index.js:382
      process.off('SIGINT', abort);
              ^
TypeError: undefined is not a function
    at enstore.<anonymous> (/usr/local/lib/node_modules/duo/node_modules/duo-package/index.js:382:15)
    at enstore.EventEmitter.emit (events.js:104:17)
    at Stream.end (/usr/local/lib/node_modules/duo/node_modules/duo-package/node_modules/enstore/index.js:35:10)
    at _end (/usr/local/lib/node_modules/duo/node_modules/duo-package/node_modules/enstore/node_modules/through/index.js:61:9)
    at Stream.stream.end (/usr/local/lib/node_modules/duo/node_modules/duo-package/node_modules/enstore/node_modules/through/index.js:70:5)
    at Gunzip.onend (_stream_readable.js:501:10)
    at Gunzip.g (events.js:199:16)
    at Gunzip.EventEmitter.emit (events.js:129:20)
    at _stream_readable.js:898:16
    at process._tickCallback (node.js:343:11)

So far I've tried:

$ rm -rf $TMPDIR/duo

but same thing. any ideas? Was anything pushed last night that could break? Maybe a sub-package with loose dependencies?

this is a blocker atm

∴ npm install --save-dev duo@latest
npm WARN deprecated [email protected]: 'native-or-bluebird' is deprecated. Please use 'any-promise' instead.
npm WARN deprecated [email protected]: graceful-fs version 3 and before will fail on newer node releases. Please update to graceful-fs@^4.0.0 as soon as possible.
-
> [email protected] install /Users/stephenmathieson/dev/src/github.com/segmentio/meta-ui/node_modules/duo/node_modules/duo-cache/node_modules/level/node_modules/leveldown
> prebuild --install

npm ERR! Darwin 15.0.0
npm ERR! argv "/Users/stephenmathieson/.nvm/versions/node/v4.2.4/bin/node" "/Users/stephenmathieson/.nvm/versions/node/v4.2.4/bin/npm" "install" "--save-dev" "duo@latest"
npm ERR! node v4.2.4
npm ERR! npm  v2.14.12

npm ERR! version not found: [email protected]
npm ERR! 
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/stephenmathieson/dev/src/github.com/segmentio/meta-ui/npm-debug.log

after renaming some stuff (and missing an @import), duo throws a super not-helpful error:

error : Error: EISDIR: illegal operation on a directory, read
    at Error (native)

something like this would be awesome instead

Browserify has a hackish way to do this, with a section inside package.json.

What would be ideal would be a pre-processor independent way of saying what is being required inside a module. E.g.,

# HappyReactComponent/package.json
{
   "codebase": "es6 jsx png myth"
}

Or maybe it should use mime types or something.

Hi, using this CLI cmd and 0.15.3

duo --watch client/main.js --use duo-babel

It builds fine, but then doesn't watch anything.

This is basically the folder structure I have after running the command from the root dir:

+root
|+client/main.js
|+build/client/main.js
|+components/
...

I must be missing s.th. Cheers!

I'm using Mac OS X El Capitan Beta with NPM version 2.14.1, and I got errors below:

$ sudo npm install -g duo

> [email protected] install /usr/local/lib/node_modules/duo/node_modules/duo-cache/node_modules/level/node_modules/leveldown
> prebuild --download

prebuild WARN install ENOENT, open '/Users/DYZ/.npm/_prebuilds/https-github.com-level-leveldown-releases-download-v1.4.1-leveldown-v1.4.1-node-v14-darwin-x64.tar.gz.8746-02270bb.tmp'
prebuild ERR! build EACCES, mkdir '/Users/DYZ/.node-gyp/0.12.4'
npm ERR! Darwin 15.0.0
npm ERR! argv "node" "/usr/local/bin/npm" "install" "-g" "duo"
npm ERR! node v0.12.4
npm ERR! npm  v2.10.1
npm ERR! code ELIFECYCLE

npm ERR! [email protected] install: `prebuild --download`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] install script 'prebuild --download'.
npm ERR! This is most likely a problem with the leveldown package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     prebuild --download
npm ERR! You can get their info via:
npm ERR!     npm owner ls leveldown
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/DYZ/npm-debug.log