This PR is related to issue #228. I tried to keep it simple and contained, but there were too many places where it the local filesystem was taken as a guarantee. So there was a lot of refactoring involved.

I tried to follow the conventions you used in the code, but if I failed to respect some convention please let me know and I will fix it.

Apart from the strong test suite of yours, I also tested the application inside a private hosted server with the ansible setup and it worked well for both the local and for the new S3 storage. The only problem that I could find is when migrating from local to S3, but I think in a later PR someone (or me) can provide a proper command to migrate files from local to S3 storage.

Once again, if you see any problems on the PR, let me know and I will fix in a manageable time, I spend almost a month working on this, so I would like to see it merged :smile_cat: .

Thanks for the awesome tool you guys are providing for the community.

Greetings, May I as for support regarding this error in repman.io ?

Error: preg_match() expects parameter 2 to be string, null given
Logs:
Reading composer.json of module/module-example-pricing (1.0.9)
Importing tag 1.0.9 (1.0.9.0)
Reading composer.json of module/module-example-pricing (1.0.8)
Importing tag 1.0.8 (1.0.8.0)
Reading composer.json of module/module-example-pricing  (1.0.7)
Importing tag 1.0.7 (1.0.7.0)
Reading composer.json of module/module-example-pricing  (1.0.6)
Importing tag 1.0.6 (1.0.6.0)
Reading composer.json of module/module-example-pricing  (1.0.5)
Importing tag 1.0.5 (1.0.5.0)
Reading composer.json of module/module-example-pricing  (1.0.4)
Importing tag 1.0.4 (1.0.4.0)
Reading composer.json of module/module-example-pricing  (1.0.3)
Importing tag 1.0.3 (1.0.3.0)
Reading composer.json of module/module-example-pricing  (1.0.2)
Importing tag 1.0.2 (1.0.2.0)
Reading composer.json of module/module-example-pricing  (1.0.1)
Importing tag 1.0.1 (1.0.1.0)
Reading composer.json of module/module-example-pricing  (1.0.0)
Importing tag 1.0.0 (1.0.0.0)
Reading composer.json of module/module-example-pricing  (master)
Importing branch master (dev-master)

https://imgur.com/a/5e4e1pZ

In #175 there was a request to show the package versions. Since that is also a very useful feature for us, I have added this functionality.

Some notes / remarks:

• I added a package details page so one can view the versions here. We could later possibly add more here (like the readme). I made anonymous organization users have access to it.
• You can get to the details page both using the dropdown menu as well as clicking the package name.
• Does showing the clipboard button make sense? For dev-* versions this is what you want but for other versions you might want to add ^ or ~ in front of it. Maybe only let one copy the version itself as well as a separate button to copy the package name?
• I'm not sure if my logic in the Package and Version entity make sense?
• Perhaps we should add a stable boolean to version so we can show the dev-* versions in a separate table?
• I modified some tests to also test if we have the correct versions, but I'm not sure how/if we should test the reference as well? We could do it for the artifact case, since these ZIP's should have versions. For the local path one we can't assume anything since it uses local git info.

Some screenshots:

#158 followup ...

the first entry is https://username:token , worked , custom gitlab entries get stuck ( and yes bin/console messenger:consume async is running )

Logs:

{"time":"2020-05-19T19:23:34.667Z","severity":"INFO","duration":234.94,"db":21.03,"view":213.91,"status":200,"method":"GET","path":"/api/v4/projects","params":[{"key":"simple","value":"true"},{"key":"order_by","value":"last_activity_at"},{"key":"owned","value":"true"}],"host":"somegitlab.tld","remote_ip":"0.0.0.0, 0.0.0.0","ua":"php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)","route":"/api/:version/projects","user_id":23,"username":"someuser","queue_duration":21.02,"gitaly_calls":1,"gitaly_duration":4.54,"correlation_id":"U7pPs8VDRH5"}
{"time":"2020-05-19T19:23:34.853Z","severity":"INFO","duration":149.55,"db":12.67,"view":136.88000000000002,"status":200,"method":"GET","path":"/api/v4/projects","params":[{"key":"membership","value":"false"},{"key":"order_by","value":"last_activity_at"},{"key":"owned","value":"true"},{"key":"page","value":"2"},{"key":"per_page","value":"20"},{"key":"simple","value":"true"},{"key":"sort","value":"desc"},{"key":"starred","value":"false"},{"key":"statistics","value":"false"},{"key":"with_custom_attributes","value":"false"},{"key":"with_issues_enabled","value":"false"},{"key":"with_merge_requests_enabled","value":"false"}],"host":"somegitlab.tld","remote_ip":"0.0.0.0, 0.0.0.0","ua":"php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)","route":"/api/:version/projects","user_id":23,"username":"someuser","queue_duration":23.96,"gitaly_calls":2,"gitaly_duration":12.32,"correlation_id":"ppcecS9JOB3"}
Started GET "/api/v4/projects?simple=true&order_by=last_activity_at&owned=true" for 0.0.0.0 at 2020-05-19 21:23:34 +0200
Started GET "/api/v4/projects?membership=false&order_by=last_activity_at&owned=true&page=2&per_page=20&simple=true&sort=desc&starred=false&statistics=false&with_custom_attributes=false&with_issues_enabled=false&with_merge_requests_enabled=false" for 0.0.0.0 at 2020-05-19 21:23:34 +0200
Started GET "/api/v4/projects?simple=true&order_by=last_activity_at&membership=true" for 0.0.0.0 at 2020-05-19 21:23:34 +0200
0.0.0.0 - - [19/May/2020:21:23:34 +0200] "GET /api/v4/projects?simple=true&order_by=last_activity_at&owned=true HTTP/2.0" 200 2347 "" "php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)"
0.0.0.0 - - [19/May/2020:21:23:34 +0200] "GET /api/v4/projects?membership=false&order_by=last_activity_at&owned=true&page=2&per_page=20&simple=true&sort=desc&starred=false&statistics=false&with_custom_attributes=false&with_issues_enabled=false&with_merge_requests_enabled=false HTTP/2.0" 200 1786 "" "php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)"
{"time":"2020-05-19T19:23:35.043Z","severity":"INFO","duration":167.44,"db":12.04,"view":155.4,"status":200,"method":"GET","path":"/api/v4/projects","params":[{"key":"simple","value":"true"},{"key":"order_by","value":"last_activity_at"},{"key":"membership","value":"true"}],"host":"somegitlab.tld","remote_ip":"0.0.0.0, 0.0.0.0","ua":"php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)","route":"/api/:version/projects","user_id":23,"username":"someuser","queue_duration":13.93,"gitaly_calls":1,"gitaly_duration":4.41,"correlation_id":"d4nM4bsioW2"}
{"time":"2020-05-19T19:23:35.476Z","severity":"INFO","duration":398.42,"db":10.77,"view":387.65000000000003,"status":200,"method":"GET","path":"/api/v4/projects","params":[{"key":"membership","value":"true"},{"key":"order_by","value":"last_activity_at"},{"key":"owned","value":"false"},{"key":"page","value":"2"},{"key":"per_page","value":"20"},{"key":"simple","value":"true"},{"key":"sort","value":"desc"},{"key":"starred","value":"false"},{"key":"statistics","value":"false"},{"key":"with_custom_attributes","value":"false"},{"key":"with_issues_enabled","value":"false"},{"key":"with_merge_requests_enabled","value":"false"}],"host":"somegitlab.tld","remote_ip":"0.0.0.0, 0.0.0.0","ua":"php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)","route":"/api/:version/projects","user_id":23,"username":"someuser","queue_duration":26.82,"gitaly_calls":2,"gitaly_duration":10.08,"correlation_id":"Bm2ICu7Fht7"}
Started GET "/api/v4/projects?membership=true&order_by=last_activity_at&owned=false&page=2&per_page=20&simple=true&sort=desc&starred=false&statistics=false&with_custom_attributes=false&with_issues_enabled=false&with_merge_requests_enabled=false" for 0.0.0.0 at 2020-05-19 21:23:35 +0200
0.0.0.0 - - [19/May/2020:21:23:35 +0200] "GET /api/v4/projects?simple=true&order_by=last_activity_at&membership=true HTTP/2.0" 200 2630 "" "php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)"
0.0.0.0 - - [19/May/2020:21:23:35 +0200] "GET /api/v4/projects?membership=true&order_by=last_activity_at&owned=false&page=2&per_page=20&simple=true&sort=desc&starred=false&statistics=false&with_custom_attributes=false&with_issues_enabled=false&with_merge_requests_enabled=false HTTP/2.0" 200 2304 "" "php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)"
{"time":"2020-05-19T19:23:36.869Z","severity":"INFO","duration":16.15,"db":3.11,"view":13.04,"status":200,"method":"GET","path":"/api/v4/projects/162/hooks","params":[],"host":"somegitlab.tld","remote_ip":"0.0.0.0, 0.0.0.0","ua":"php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)","route":"/api/:version/projects/:id/hooks","user_id":23,"username":"someuser","queue_duration":12.03,"correlation_id":"juHkkUtLjS8"}
{"time":"2020-05-19T19:23:36.936Z","severity":"INFO","duration":32.77,"db":5.28,"view":27.490000000000002,"status":201,"method":"POST","path":"/api/v4/projects/162/hooks","params":[{"key":"push_events","value":"1"},{"key":"tag_push_events","value":"1"},{"key":"url","value":"https://repman.some.where.else/hook/82d60d69-edb7-434b-8f35-d60fb5326e85"}],"host":"somegitlab.tld","remote_ip":"0.0.0.0, 0.0.0.0","ua":"php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)","route":"/api/:version/projects/:id/hooks","user_id":23,"username":"someuser","queue_duration":13.79,"correlation_id":"g7KNez7wdk3"}
Started GET "/api/v4/projects/162/hooks" for 0.0.0.0 at 2020-05-19 21:23:36 +0200
Started POST "/api/v4/projects/162/hooks" for 0.0.0.0 at 2020-05-19 21:23:36 +0200
0.0.0.0 - - [19/May/2020:21:23:36 +0200] "GET /api/v4/projects/162/hooks HTTP/2.0" 200 2 "" "php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)"
0.0.0.0 - - [19/May/2020:21:23:36 +0200] "POST /api/v4/projects/162/hooks HTTP/2.0" 201 492 "" "php-gitlab-api (http://github.com/m4tthumphrey/php-gitlab-api)"

Hi !

I've added a package from a repository where Repman do not have access, and the sync is stuck, and there is no way to delete the package.

Have a good day, Samuel

Hi, thanks guys for a good app.

I am playing with your app and planning to implement it in our Development Flow.

We have multiple teams, which should be fully isolated from each other, except a few shared packages. We also have a vendor, which has multiple private packages, which should be shared for everyone authenticated.

Every developer has a personal OpenLDAP account, but we use OpenID Connect to authenticate (SSO).
At the bottom there is my list of things I need to have to launch in production., can do you have any plans to add such a feature?

1. I can split teams as a different organizations in the current situation. Can a user be in multiple organization/teams? We need this.
2. LDAP Users
3. OpenID Connect
4. Sync vendor private repository
5. Disable LDAP Auth ( It can be useful to prevent using LDAP credentials through the web UI when an alternative such as OpenID Connect is preferred. This allows LDAP to be used for group sync, while also allowing your OpenID Connect identity provider to handle additional checks like custom 2FA. and protects from auth by deleted user.
6. Does it sync dev branches?
7. org.repo.repo.corp.xxx.xxx - it is a very long domain name. Can we use folders per organisation? Can repo domain prefix be configured?
8. Sync support of a hosted bitbucket?

• [x] A copy button for the package name (makes installation much easier)
• [x] Ability to view the README for a package
• [x] List of available releases
• [ ] List of dependencies

Mimics our commonly used features from satis/packagist

I've got access to a lot more than 40 repos in GitLab, but only 40 appear in the packages list when trying to add a new package. I assume this is because of an issue with the pagination in the GitLab API, but because there's no way to manually add a repo, I'm unable to add any repos that don't appear in those 40.

Show package dependencies in detail view, linking to other packages known in the same organisation.

Allow searching for other packages that depend on this package (count is shown on the details view):

Fixes #175

composer install fails due to errors with packages hosted on repman.io. The log shows the following:

  - Installing messengerpeople/specification-database (v1.1.1): Downloading
 Failed, trying the next URL (404: The "https://ourhandle.repo.repman.io/dists/ourhandle/packagename/1.1.1.0/a8b7d5c6d095fb50e07fc5e420f45164513902ca.tar" file could not be downloaded (HTTP/1.1 404 Not Found))Downloading    Failed to download ourhandle/packagename from dist: The "https://api.github.com/repos/ourhandle/packagename/zipball/a8b7d5c6d095fb50e07fc5e420f45164513902ca" file could not be downloaded (HTTP/1.1 404 Not Found)

This is probably similar to #84, but our versions never contain slashes but look like v1.2.3 instead. It's also similar to #67, but the error does occur for the repman.io sources too, not only for GitHub.
As @akondas pointed out in #67, we do have checksums in the lock file, but composer seems to insert it automatically.

As we have multiple engineers working together, I suspect one of them has an older composer version that causes the "broken" lockfile. Specifically, the last one who committed a lockfile change in one of our projects had 1.8.0 installed. His lockfile looked like this:

{
    "name": "ourhandle/packagename",
    "version": "v1.1.1",
    "source": {
        "type": "git",
        "url": "[email protected]:ourhandle/packagename.git",
        "reference": "a8b7d5c6d095fb50e07fc5e420f45164513902ca"
    },
    "dist": {
        "type": "tar",
        "url": "https://api.github.com/repos/ourhandle/packagename/zipball/a8b7d5c6d095fb50e07fc5e420f45164513902ca",
        "reference": "a8b7d5c6d095fb50e07fc5e420f45164513902ca",
        "shasum": "e098ed99b0e445508105b9256d15baf648c9b907",
        "mirrors": [
            {
                "url": "https://ourhandle.repo.repman.io/dists/%package%/%version%/%reference%.%type%",
                "preferred": true
            }
        ]
    },

Executing composer install with this file causes the installation to fail due to the above mentioned error. If I do composer update however, the lockfile is updated to the following:

{
    "name": "ourhandle/packagename",
    "version": "v1.1.1",
    "source": {
        "type": "git",
        "url": "[email protected]:ourhandle/packagename.git",
        "reference": "a8b7d5c6d095fb50e07fc5e420f45164513902ca"
    },
    "dist": {
        "type": "zip",
        "url": "https://api.github.com/repos/ourhandle/packagename/zipball/a8b7d5c6d095fb50e07fc5e420f45164513902ca",
        "reference": "a8b7d5c6d095fb50e07fc5e420f45164513902ca",
        "shasum": "",
        "mirrors": [
            {
                "url": "https://ourhandle.repo.repman.io/dists/%package%/%version%/%reference%.%type%",
                "preferred": true
            }
        ]
    },

Diff being the dist.type being zip instead of tar and dist.shasum being empty in the working configuration.

Sorry for the long issue. So the actual question is - was there any known change in checksum handling in newer composer versions? And by extension, is repman.io incompatible with older versions? That would be an interesting requirement to show in the Readme.

Hi guys

I've been trying to add packages via the API. Authentication works and and the POST request goes through, but in the Repman UI it shows the following error:

SYNCHRONISATION ERROR
Error: Failed to execute git clone --mirror '[email protected]:my/repo.git' '/home/repman/.composer/cache/vcs/git-gitlab.com-my-repo.git/'

Cloning into bare repository '/home/repman/.composer/cache/vcs/git-gitlab.com-my-repo.git'...
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Logs:
Failed to clone the [email protected]:my/repo.git repository, try running in interactive mode so that you can enter your credentials

However, when I try and add the package manually (via the UI) everything works. So I'm assuming it's some kind of bug with the API?

Cheers

Hello,

we are selling modules from our website

now we are trying to support the install module using the composer

I have a question about that : customer x buy package y, so i want to give access only y package for customer X

i don't want to give another package access

I checked your api document but i did not find any API for same, can you please help us on that?

Hello,

I've tried to install repman now for a few hours in our docker environment and I gave up.

There are several issues with the permissions. The application in the initial setup doesn't work,. Giving all folders the full permissions gets the app to work (since I don't get any log entries I don't know what the issue is).

Reproducing:

• git clone https://github.com/repman-io/repman.git (f25582a9327d0b8781afac16540d7aadaa8f194c)
• cd repman
• docker-compose up -d
• browse url: https://x.x.x.x -- 500 Internal Server Error

Docker Version: Docker version 20.10.22, build 3a2c30b

Kind regards Philip

Hello. I deployed my self-hosted repman yesterday and I am starting to use.

I am trying to invite a member to my organization, but I am getting a 500 error on POST function.

repman_access.log: 10.0.142.25 - - [28/Oct/2022:19:18:34 +0000] "POST /organization/xxxxxxxx/member/invite HTTP/1.1" 500 5343 "https://xxxxxx.com.br/organization/xxxxxxxxx/member/invite" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/105.0" "177.92.89.182" 0.352

Prod.log: [2022-10-28T19:18:34.068853+00:00] request.INFO: Matched route "organization_invite_member". {"route":"organization_invite_member","route_parameters":{"_route":"organization_invite_member","_controller":"Buddy\\Repman\\Controller\\Organization\\MembersController::invite","organization":"xxxxxxxx"},"request_uri":"https://xxxxxxxxxx.com.br/organization/xxxxxxxxxxx/member/invite","method":"POST"} [] [2022-10-28T19:18:34.087226+00:00] security.DEBUG: Read existing security token from the session. {"key":"_security_main","token_class":"Symfony\\Component\\Security\\Guard\\Token\\PostAuthenticationGuardToken"} [] [2022-10-28T19:18:34.129149+00:00] security.DEBUG: User was reloaded from a user provider. {"provider":"Buddy\\Repman\\Security\\UserProvider","username":"[email protected]"} [] [2022-10-28T19:18:34.129181+00:00] security.DEBUG: Checking for guard authentication credentials. {"firewall_key":"main","authenticators":5} [] [2022-10-28T19:18:34.129191+00:00] security.DEBUG: Checking support on guard authenticator. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\LoginFormAuthenticator"} [] [2022-10-28T19:18:34.129201+00:00] security.DEBUG: Guard authenticator does not support the request. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\LoginFormAuthenticator"} [] [2022-10-28T19:18:34.144075+00:00] security.DEBUG: Checking support on guard authenticator. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\GitHubAuthenticator"} [] [2022-10-28T19:18:34.144092+00:00] security.DEBUG: Guard authenticator does not support the request. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\GitHubAuthenticator"} [] [2022-10-28T19:18:34.144248+00:00] security.DEBUG: Checking support on guard authenticator. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\GitLabAuthenticator"} [] [2022-10-28T19:18:34.144254+00:00] security.DEBUG: Guard authenticator does not support the request. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\GitLabAuthenticator"} [] [2022-10-28T19:18:34.146141+00:00] security.DEBUG: Checking support on guard authenticator. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\BitbucketAuthenticator"} [] [2022-10-28T19:18:34.146157+00:00] security.DEBUG: Guard authenticator does not support the request. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\BitbucketAuthenticator"} [] [2022-10-28T19:18:34.153410+00:00] security.DEBUG: Checking support on guard authenticator. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\BuddyAuthenticator"} [] [2022-10-28T19:18:34.153429+00:00] security.DEBUG: Guard authenticator does not support the request. {"firewall_key":"main","authenticator":"Buddy\\Repman\\Security\\BuddyAuthenticator"} [] [2022-10-28T19:18:34.223658+00:00] php.INFO: User Deprecated: Since symfony/http-foundation 5.3: "Symfony\Component\HttpFoundation\RequestStack::getMasterRequest()" is deprecated, use "getMainRequest()" instead. {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since symfony/http-foundation 5.3: \"Symfony\\Component\\HttpFoundation\\RequestStack::getMasterRequest()\" is deprecated, use \"getMainRequest()\" instead. at /app/vendor/symfony/http-foundation/RequestStack.php:92)"} [] [2022-10-28T19:18:34.310722+00:00] app.DEBUG: Email transport "Symfony\Component\Mailer\Transport\Smtp\SmtpTransport" starting [] [] [2022-10-28T19:18:34.332182+00:00] request.CRITICAL: Uncaught PHP Exception Symfony\Component\Messenger\Exception\HandlerFailedException: "Connection could not be established with host "mailhog:1025": stream_socket_client(): php_network_getaddresses: getaddrinfo failed: Name does not resolve" at /app/vendor/symfony/doctrine-bridge/Messenger/DoctrineTransactionMiddleware.php line 42 {"exception":"[object] (Symfony\\Component\\Messenger\\Exception\\HandlerFailedException(code: 0): Connection could not be established with host \"mailhog:1025\": stream_socket_client(): php_network_getaddresses: getaddrinfo failed: Name does not resolve at /app/vendor/symfony/doctrine-bridge/Messenger/DoctrineTransactionMiddleware.php:42)\n[previous exception] [object] (Symfony\\Component\\Messenger\\Exception\\HandlerFailedException(code: 0): Connection could not be established with host \"mailhog:1025\": stream_socket_client(): php_network_getaddresses: getaddrinfo failed: Name does not resolve at /app/vendor/symfony/doctrine-bridge/Messenger/DoctrineTransactionMiddleware.php:42)\n[previous exception] [object] (Symfony\\Component\\Mailer\\Exception\\TransportException(code: 0): Connection could not be established with host \"mailhog:1025\": stream_socket_client(): php_network_getaddresses: getaddrinfo failed: Name does not resolve at /app/vendor/symfony/mailer/Transport/Smtp/Stream/SocketStream.php:139)"} [] [2022-10-28T19:18:34.340752+00:00] php.INFO: User Deprecated: Since symfony/http-foundation 5.3: "Symfony\Component\HttpFoundation\RequestStack::getMasterRequest()" is deprecated, use "getMainRequest()" instead. {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since symfony/http-foundation 5.3: \"Symfony\\Component\\HttpFoundation\\RequestStack::getMasterRequest()\" is deprecated, use \"getMainRequest()\" instead. at /app/vendor/symfony/http-foundation/RequestStack.php:92)"} []

Someone can help?

Thanks!

Bumps symfony/validator from 5.4.2 to 5.4.13.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Any request i make against the API be it through curl or postman (adding X-API-TOKEN to header) returns invalid credentials.

Even using the preconfigured requests at https://repman.io/docs/api/ Still returns:

I am authorising with a correct API token which exists for my organisation, So unsure why it's being denied.

"Allow anonymous users" is disabled.

This fixes the issue in #562 where if the first fetched package is not stable, then no stable release can be determined because the version comparison will fail with Comparator::greaterThan.