Transaction limit?

#1

Hello there,

I just started using the CLI apid, so I might be missing something, but it looks like (at least in release 0.6) the program is limited to 15 transactions. I only have one step per transaction which is to send a GET request and expect a 200 OK status code, and each transaction sends that for a different URL. If I add a similar 16th transaction to my config file, the program just hangs with no output. Am I doing something wrong?

#2

Hi @etienne,

There is no limit in the number of transactions or steps per transaction. Can you please share the config file you’re trying to execute?

Thanks,
Ivo

#3

I’ve found the issue, there seems to be a deadlock with the channels that hold transactions for execution. This has now been resolved in v0.6.1.

Please do give it a go!

Thanks,
Ivo

#4 
% ./apid version
APId 0.6.1
% ./apid check
…
successful transactions: 63/88
failed transactions:     25/88

Thank you!

1 Like
#5 
% ./apid version   
APId 0.6.1
% ./apid check
…
successful transactions: 869/1352
failed transactions:     483/1352

:heart_eyes: Definitely fixed, thank you so much! Now these 1352 transactions took about 20mn to complete on a low-cost VM. I believe the software runs them sequentially. Is that correct? And if so, do you have plans on letting your program run them in parallel? Maybe a command-line option could be used to define how many transactions to run in parallel, similarly to “-j” in make(1)?

#6

Hi @etienne,

I’m really glad you’re enjoying APId ^^.

The transactions were executed in sequence up until recently but that has changed in v0.6. This is actually where the issue fixed in v0.6.1 came from. At the moment there is a fixed number of 5 threads that run the transactions, which is hard coded.

I’ll draft a change to expose this and try to get it released asap.

Thanks,
Ivo

1 Like
#7

Hi @etienne !

As of v0.6.2 (which will be released once github sort the issues with actions :smiley: ) you can specify the number of parallel transaction executions with the --parallelism (or -p) flag when running apid :slight_smile:

apid -c /config_file.yaml -p 10

Let me know if there’s anything else you find missing or need adding!

Have a wonderful day!
Ivo

1 Like
#8

Mate, you rock! I ran the same file with “-p 100”, and that got the job done in less than 3mn (all those URLs that timed out had to be waited for, but if all were responding it would have been much shorter). Meanwhile, the load hardly changed and the apid process peaked at 6% of the CPU time.

% time ./apid check -p 100 -c apid.yaml
…
successful transactions: 868/1352
failed transactions:     484/1352
./apid check -p 100 -c apid.yaml  3.06s user 0.72s system 2% cpu 2:40.71 total

Great stuff! Again, thank you!

Oh, and I need to mention that releasing binaries for every platform Go is supported on is really nice. I ran this above on OpenBSD, and having a binary release made my life much easier.

1 Like