switch to OSDN for iso hosting

philm commented

2016-07-01 23:10:50 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

This looks good, it doesn't state that a project needs to start the shell server manually like on SF.
It seems to be available if a project is hosted there.

https://en.osdn.jp/docs/ShellServer

*Created by: udeved* This looks good, it doesn't state that a project needs to start the shell server manually like on SF. It seems to be available if a project is hosted there. https://en.osdn.jp/docs/ShellServer

philm commented

2016-07-02 08:55:44 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: philmmanjaro

I created a project called manjaro. Will create another one called manjaro-spins.

*Created by: philmmanjaro* I created a project called **manjaro**. Will create another one called **manjaro-spins**.

philm commented

2016-07-03 21:21:12 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: philmmanjaro

Seems you can only upload with sftp. You have ssh for homepage management. The structure however is totally different. With sf.net you have [more options](https://sourceforge.net/p/forge/documentation/File Management/) to manage your files. Using rsync there is no shell server needed.

Folder structure can be created before, which seems to be the only handycap here. sshpass can be used if you use a bash script to upload your files without a password request. If not used, the password gets asked to type in. Since OSDN doesn't support rsync upload yet, I see no option to follow up this further ...

*Created by: philmmanjaro* Seems you can only upload with **sftp**. You have **ssh** for **homepage** management. The structure however is totally different. With **sf.net** you have [more options](https://sourceforge.net/p/forge/documentation/File Management/) to manage your files. Using **rsync** there is no shell server needed. Folder structure can be created before, which seems to be the only handycap here. **sshpass** can be used if you use a bash script to upload your files without a password request. If not used, the password gets asked to type in. Since OSDN doesn't support **rsync upload** yet, I see no option to follow up this further ...

philm commented

2016-07-03 21:26:58 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

ah, ok.

Yes, SF is ok, but, folder creation is ugly with rsync as workaround to ssh shell, which is available, but you need to start it manually, it lasts iirc 1 hour.
Can you also delete projects on SF? There are too many projects with manjaro hosting iso on SF.

*Created by: udeved* ah, ok. Yes, SF is ok, but, folder creation is ugly with rsync as workaround to ssh shell, which is available, but you need to start it manually, it lasts iirc 1 hour. Can you also delete projects on SF? There are too many projects with manjaro hosting iso on SF.

philm commented

2016-07-03 21:32:04 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: philmmanjaro

No, I can only delete projects which I own. Which is manjarolinux, manjarodev and manjarotest.

*Created by: philmmanjaro* No, I can only delete projects which I own. Which is [manjarolinux](https://sourceforge.net/projects/manjarolinux/), [manjarodev](https://sourceforge.net/projects/manjarodev/) and [manjarotest](https://sourceforge.net/projects/manjarotest/).

philm commented

2016-07-03 21:36:32 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

I also have 3 basically play projects on SF with manjaro in name. But how/where can I delete it on SF?

*Created by: udeved* I also have 3 basically play projects on SF with manjaro in name. But how/where can I delete it on SF?

philm commented

2016-07-03 21:39:40 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: philmmanjaro

Regarding the shell. You need it only to get the file structure done. That I do mostly in front with the webservices directly on SF.net as I can hide the folder for 3 days. Then I modify my release script and use that for uploading:

#!/bin/sh
user=johndoe
project=foo
path=bar/1.0
files=foobar-1.0*
pwfile=./passwd
sshpass -f $pwfile rsync -vP  --progress -e ssh $files $user,$project@frs.sourceforge.net:/home/frs/project/$project/$path

*Created by: philmmanjaro* Regarding the **shell**. You need it only to get the file structure done. That I do mostly in front with the webservices directly on SF.net as I can hide the folder for 3 days. Then I modify my release script and use that for uploading: ``` #!/bin/sh user=johndoe project=foo path=bar/1.0 files=foobar-1.0* pwfile=./passwd sshpass -f $pwfile rsync -vP --progress -e ssh $files $user,$project@frs.sourceforge.net:/home/frs/project/$project/$path ```

philm commented

2016-07-03 21:42:05 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

deployiso does the same but with ssh keys instead of pwd files, and creates eventually missing folders with a switch with rsync.
But folder creation would be much better with ssh shell, because rsync can only automatically create one child folder in the parent.
For each subfolder, one rsync call, ugly.

*Created by: udeved* deployiso does the same but with ssh keys instead of pwd files, and creates eventually missing folders with a switch with rsync. But folder creation would be much better with ssh shell, because rsync can only automatically create one child folder in the parent. For each subfolder, one rsync call, ugly.

philm commented

2016-07-03 21:46:06 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: philmmanjaro

Ok, so in which case the shell is needed then? Regarding deletion of a project, you have to talk to a staff member.

*Created by: philmmanjaro* Ok, so in which case the **shell** is needed then? Regarding deletion of a project, you have to talk to a [staff member](https://sourceforge.net/p/forge/documentation/How%20to%20Remove%20a%20Project/).

philm commented

2016-07-03 21:50:04 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

No, current state does missing subfolders with rsync on switch, but each subfolder one rsync call.
Ideally, we create folders with shell, and then rsync the files or folder.
There is no way without shell to check for folder existence.

create_subtree(){
    msg2 "Create (%s) ..." "${edition}/$1/${dist_release}"
    rsync ${rsync_args[*]} /dev/null ${sf_url}/${edition}/
    rsync ${rsync_args[*]} /dev/null ${sf_url}/${edition}/$1/
    rsync ${rsync_args[*]} /dev/null ${sf_url}/${edition}/$1/${dist_release}/
    msg2 "Done"
    show_elapsed_time "${FUNCNAME}" "${timer_start}"
}

ugly, and on switch, pretty static.

*Created by: udeved* No, current state does missing subfolders with rsync on switch, but each subfolder one rsync call. Ideally, we create folders with shell, and then rsync the files or folder. There is no way without shell to check for folder existence. ``` create_subtree(){ msg2 "Create (%s) ..." "${edition}/$1/${dist_release}" rsync ${rsync_args[*]} /dev/null ${sf_url}/${edition}/ rsync ${rsync_args[*]} /dev/null ${sf_url}/${edition}/$1/ rsync ${rsync_args[*]} /dev/null ${sf_url}/${edition}/$1/${dist_release}/ msg2 "Done" show_elapsed_time "${FUNCNAME}" "${timer_start}" } ``` ugly, and on switch, pretty static.

philm commented

2016-07-03 22:01:32 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

This is the unused code, if there was a permanent shell access, that could be run always, no extra switch.

create_subtree_ssh(){
    local tree=${remote_target}/${remote_project}/${remote_dir}
        ssh !${remote_user}@${shell_url} [[ ! -d $tree ]] && mkdir -pv $tree
}

For each rsync call, one login is used, advantage ssh.

*Created by: udeved* This is the unused code, if there was a permanent shell access, that could be run always, no extra switch. ``` create_subtree_ssh(){ local tree=${remote_target}/${remote_project}/${remote_dir} ssh !${remote_user}@${shell_url} [[ ! -d $tree ]] && mkdir -pv $tree } ``` For each rsync call, one login is used, advantage ssh.

philm commented

2016-07-04 09:05:52 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

Would it be better to spilt the deployment in a separate package that depends on ruby?

*Created by: udeved* Would it be better to spilt the deployment in a separate package that depends on ruby?

philm commented

2016-07-04 19:22:00 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: philmmanjaro

You mean deployiso? Regarding Sado's email you were CCed in?

https://osdn.jp/docs/FileRelease_Guide
https://osdn.jp/projects/osdn-codes/wiki/APIGuide
https://osdn.jp/projects/osdn-codes/wiki/CommandLineInterface

Let us test it first how it works. Maybe we are on to something here. If we need some, Sado told me that the development team will implement it.

*Created by: philmmanjaro* You mean **deployiso**? Regarding Sado's email you were CCed in? https://osdn.jp/docs/FileRelease_Guide https://osdn.jp/projects/osdn-codes/wiki/APIGuide https://osdn.jp/projects/osdn-codes/wiki/CommandLineInterface Let us test it first how it works. Maybe we are on to something here. If we need some, Sado told me that the development team will implement it.

philm commented

2016-07-09 19:24:32 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

I have added a ruby-osdn-cli package.

https://github.com/manjaro/packages-extra/tree/master/ruby-osdn-cli

This installs the osdn-cli gem systemwide, I created also a osdn branch I'll push soon.
The downside, it amounts to 7 ruby gem packages in total to get osdn cli working.

*Created by: udeved* I have added a ruby-osdn-cli package. https://github.com/manjaro/packages-extra/tree/master/ruby-osdn-cli This installs the osdn-cli gem systemwide, I created also a osdn branch I'll push soon. The downside, it amounts to 7 ruby gem packages in total to get osdn cli working.

philm commented

2016-07-10 08:58:48 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

buildlist:

pkg.list.d/osdn.list

ruby-ffi
ruby-ethon
ruby-typhoeus
ruby-hashie
ruby-json
ruby-osdn-client
ruby-osdn-cli

*Created by: udeved* buildlist: pkg.list.d/osdn.list ``` ruby-ffi ruby-ethon ruby-typhoeus ruby-hashie ruby-json ruby-osdn-client ruby-osdn-cli ```

philm commented

2016-07-10 10:05:34 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

https://osdn.jp/projects/deployment-test/

This is my testing and evaluation account.

*Created by: udeved* https://osdn.jp/projects/deployment-test/ This is my testing and evaluation account.

philm commented

2016-07-10 19:22:34 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

Tested ruby gem packages

x86_64
i686

$ osdn help
/usr/bin/osdn [global-options] <command> [command-options] [args]
/usr/bin/osdn help <command>
Global Options:
  -h --help      Show help message. use 'help <command>' for specific command. 
  -v --verbose   Increase log level (multiple)
  -q --quiet     Decrease log level (multiple)
Avaiable Commands:
  help
  login          Login and save access token.
  package        Manipulate frs packages of project
  release        Manipulate frs releases of project
  relfile        Manipulate frs files of project
  frs_mkdirs     Make directory tree for current project release
  frs_upload     Upload local file tree and create package/release implicitly.
  ping           Test API request.
  vars           Get/set request environment variable.

*Created by: udeved* Tested ruby gem packages - [x] x86_64 - [x] i686 ``` $ osdn help /usr/bin/osdn [global-options] <command> [command-options] [args] /usr/bin/osdn help <command> Global Options: -h --help Show help message. use 'help <command>' for specific command. -v --verbose Increase log level (multiple) -q --quiet Decrease log level (multiple) Avaiable Commands: help login Login and save access token. package Manipulate frs packages of project release Manipulate frs releases of project relfile Manipulate frs files of project frs_mkdirs Make directory tree for current project release frs_upload Upload local file tree and create package/release implicitly. ping Test API request. vars Get/set request environment variable. ```

philm commented

2016-07-11 05:36:33 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: philmmanjaro

Can this tool resume the upload if for example the connection breaks down?

*Created by: philmmanjaro* Can this tool resume the upload if for example the connection breaks down?

philm commented

2016-07-11 08:39:49 +00:00

(Migrated from gitlab2.manjaro.org)

Created by: udeved

Haven't found info on that.
I tested a bit yesterday osdn-cli.

It has its downside, but I read on osdn, that they also offer ssh and rsync, which I would prefer.
osdn-cli seems to be pretty new tool, and perhaps not feature complete.
I missed bandwidth control.
I also find the osdn login command not so nice, because I need to manually enter a generated token, no idea how to automatically get this token yet.
But, the login token expires after 24 hours, which is nice.

We would also need to come up with different server directory structure. The osdn can't handle more than two levels depth of directories.

*Created by: udeved* Haven't found info on that. I tested a bit yesterday osdn-cli. It has its downside, but I read on osdn, that they also offer ssh and rsync, which I would prefer. osdn-cli seems to be pretty new tool, and perhaps not feature complete. I missed bandwidth control. I also find the osdn login command not so nice, because I need to manually enter a generated token, no idea how to automatically get this token yet. But, the login token expires after 24 hours, which is nice. We would also need to come up with different server directory structure. The osdn can't handle more than two levels depth of directories.

Rows
Columns

switch to OSDN for iso hosting #236