pacman-mirrors should be asynchronous

joeky888 commented

2018-12-31 05:53:48 +00:00

(Migrated from gitlab2.manjaro.org)

The command "pacman-mirrors -f 0" takes too much time, it should be executed asynchronously.

The command `"pacman-mirrors -f 0"` takes too much time, it should be executed asynchronously.

ghost commented

2018-12-31 06:53:01 +00:00

(Migrated from gitlab2.manjaro.org)

You are right - depending on the speed of the network connection and the number of mirrors in the pool, it can take considerable time to probe them.

Several factors are taken into account when probing the mirrors using the -f.

mirrors which has a status of 0 for the system's branch are ignored
mirrors which has a status of -1 (not responding) are ignored

This will reduce the number of mirrors actually probed, since it very rarely all mirrors are up-to-date for the system branch.

But if your connection is very slow then probing even 10 mirrors can feel like a long time.
Running the command in a terminal does not require you to watch it run - you can put the terminal on another desktop and watch a movie while pacman-mirrors does its job.

You can also change which file is used to measure the mirrors response time. You can use any file from the core repo - just uncomment the line pacman-mirrrors.conf and supply an existing filename.

But the idea is good - let me start thinking.

You are right - depending on the speed of the network connection and the number of mirrors in the pool, it can take considerable time to probe them. Several factors are taken into account when probing the mirrors using the -f. * mirrors which has a status of **0** for the system's branch are ignored * mirrors which has a status of **-1** (not responding) are ignored This will reduce the number of mirrors actually probed, since it very rarely all mirrors are up-to-date for the system branch. But if your connection is very slow then probing even 10 mirrors can feel like a long time. Running the command in a terminal does not require you to watch it run - you can put the terminal on another desktop and watch a movie while pacman-mirrors does its job. You can also change which file is used to measure the mirrors response time. You can use any file from the **core** repo - just uncomment the line **pacman-mirrrors.conf** and supply an existing filename. --- But the idea is good - let me start thinking. ![thinking-hat](/uploads/83a24a74fc2f3a2bd7409a37f7ff185a/thinking-hat.png)

ghost commented

2018-12-31 06:53:32 +00:00

(Migrated from gitlab2.manjaro.org)

assigned to @fhdk

joeky888 commented

2018-12-31 07:08:16 +00:00

(Migrated from gitlab2.manjaro.org)

and watch a movie while pacman-mirrors does its job.

lol, one more question, why the default branch of this repo is 3.0.x-stable not master?

> and watch a movie while pacman-mirrors does its job. lol, one more question, why the default branch of this repo is `3.0.x-stable` not `master`?

ghost commented

2018-12-31 07:11:38 +00:00

(Migrated from gitlab2.manjaro.org)

I don't know - it has confused me more than once.

ghost commented

2019-04-26 08:32:50 +00:00

(Migrated from gitlab2.manjaro.org)

mentioned in commit 77e3fffc39

mentioned in commit 77e3fffc3921738b4203b623bfd004220699d457

ZenTauro commented

2019-05-16 20:03:14 +00:00

(Migrated from gitlab2.manjaro.org)

Hi, I'm currently working on that feature, where should I push the changes whenever I have them?

ghost commented

2019-05-17 08:06:23 +00:00

(Migrated from gitlab2.manjaro.org)

So am I 😄

If you clone from my github project then I accept PR and can import it to this project

So am I :smile: If you clone from my [github project](https://github.com/fhdk/pacman-mirrors) then I accept PR and can import it to this project

ZenTauro commented

2019-05-17 13:13:51 +00:00

(Migrated from gitlab2.manjaro.org)

@fhdk , I've pushed the changes here, would you mind taking a look?

Notes:

I am not sure how to properly share the counter in test_mirror_pool across workers
Since the servers are polled asynchronously, printing the server being polled and then the response time doesn't make a lot of sense
I get a lot of timestamp too large to convert to C _PyTime_t errors
And many ResourceWarning: unclosed <socket.socket ...

@fhdk , I've pushed the changes [here](https://github.com/ZenTauro/pacman-mirrors/tree/issue144-async), would you mind taking a look? Notes: - I am not sure how to properly share the counter in `test_mirror_pool` across workers - Since the servers are polled asynchronously, printing the server being polled and then the response time doesn't make a lot of sense - I get a lot of `timestamp too large to convert to C _PyTime_t` errors - And many `ResourceWarning: unclosed <socket.socket ...`

ghost commented

2019-10-18 09:03:42 +00:00

(Migrated from gitlab2.manjaro.org)

I accept the idea is good.

Thank you for your implemententation - It is great.

The execution time and the gains - those are hard to ignore.

It seems though - in some setups/scenarios - the gain is not worth the pain - if you get the 😄

I accept the option being available using an argument - for now - with an added disclaimer for the specific argument.

I accept the idea is good. Thank you for your implemententation - It is great. The execution time and the gains - those are hard to ignore. It seems though - in some setups/scenarios - the gain is not worth the pain - if you get the :smile: I accept the option being available using an argument - for now - with an added disclaimer for the specific argument.

ghost commented

2019-10-18 09:03:43 +00:00

(Migrated from gitlab2.manjaro.org)

closed

ZenTauro commented

2019-10-18 14:01:00 +00:00

(Migrated from gitlab2.manjaro.org)

Should I implement the async flag? with something like --async to enable it?

Should I implement the async flag? with something like `--async` to enable it?

ghost commented

2019-10-19 09:44:14 +00:00

(Migrated from gitlab2.manjaro.org)

The package pacman-mirrors-dev has this flag implemented.

The package `pacman-mirrors-dev` has this flag implemented.

ghost commented

2019-10-19 12:02:43 +00:00

(Migrated from gitlab2.manjaro.org)

Also v4.15 --test-async

Also v4.15 `--test-async`

Rows
Columns

pacman-mirrors should be asynchronous #144