A new kind of Progress Bar, with real-time throughput, ETA, and very cool animations!
Have you ever wondered where your lengthy processing was at, and when would it finish? Do you usually hit RETURN
several times to make sure it didn't crash, or the SSH connection didn't freeze? Have you ever thought it'd be awesome to be able to pause some processing without hassle, return to the Python prompt to manually fix some items, then seamlessly resume it? I did...
I've started this new progress bar thinking about all that, behold the alive-progress! 😃
Introducing the newest concept in progress bars for Python! alive-progress
is in a class of its own, with an array of cool features that set it apart. Here are a few highlights:
alive_bar
widgets are kept as they were, and the elapsed time nicely ignores the paused time!check()
tool to help you design your own animations! You can see all the generated frames and cycles exploded on screen, with several verbosity levels, even including an alive rendition! It's boundless creativity at your fingertips!This README is always evolving, so do take a more comprehensive look from time to time... You might find great new details in other sections! 😊
A very cool update here! In addition to polishing things up and improving terminal support, now alive-progress
supports resuming computations!
When processing huge datasets or things that take a long time, you might either use batches or cache partial results. Then, in case it stops and is restarted, you end up skipping all those already done items very quickly, which makes the alive_bar
think you're processing thousands of items per second, which in turn completely ruins the ETA... But not anymore! Just tell bar()
that you've skipped items... 👏
You can use it in two ways:
1. If you do know where you've stopped:
with alive_bar(120000) as bar:
bar(60000, skipped=True)
for i in range(60000, 120000):
# process item
bar()
Yep, just call bar(N, skipped=True)
once, with the number of items.
2. If you do not know or the items are scattered:
with alive_bar(120000) as bar:
for i in range(120000):
if done(i):
bar(skipped=True)
continue
# process item
bar()
Yep, it's as simple as that! Just call bar(skipped=True)
when an item is already done, or bar()
as usual otherwise. You could also share a single bar(skipped=?)
call at the end, with a bool saying whether you did skip that item or not. Cool, huh?
Also in this version:
max_cols
config setting, the number of columns to use if not possible to fetch it, like in jupyter and other platforms which doesn't support sizeYep, I could finally get this version out! These are the new goodies:
B
, bytes
, or even °C
!sys.stderr
and other files instead of sys.stdout
!Highly anticipated fixes:
TypeError: unhashable type: 'types.SimpleNamespace'
.RotatingFileHandler
s! Yep, seek support is here.And last but not least, a more polished layout for you to enjoy your progress!
Now, alive_bar
supports Dual Line text mode!
If you ever wanted to include longer situational messages within the bar, you probably felt squeezed into one line. You had to shrink the beautifully animated bar or, even worse, remove widgets (!) to be able to see what you needed...
Not anymore!! You can now make the bar Dual Line, and put text below it!
Yes, there's a message below the whole bar, and any other print/logging messages scroll above it!
letters = [chr(ord('A') + x) for x in range(26)]
with alive_bar(26, dual_line=True, title='Alphabet') as bar:
for c in letters:
bar.text = f'-> Teaching the letter: {c}, please wait...'
if c in 'HKWZ':
print(f'fail "{c}", retry later')
time.sleep(0.3)
bar()
on 7: fail "H", retry later
on 10: fail "K", retry later
Alphabet |███████████████████████████▊ | ▃▅▇ 18/26 [69%] in 6s (3.2/s, eta: 3s)
-> Teaching the letter: S, please wait...
There's also a new finalize
function parameter in alive_it
which enables you to set the title and/or text of the final receipt, and improved logging support which detects customized loggers.
This is all about customization; the core widgets can now be changed:
monitor
, elapsed
, and stats
widgets to make them look anyway you want!It's incredible that these strings support all Python format features, so you can e.g.,
{percent:.1%}
😉.
They can be further customized when on the final receipt!
monitor_end
, elapsed_end
, and stats_end
, with dynamic formats inherited from the standard ones!If you've hidden some widgets before, just so they wouldn't appear on the receipt, now you can see them in all their running glory, and hide just the receipt ones! Or the other way around 😜
Another addition, now alive-progress
beautifully renders its cool final receipt whenever it is stopped, even if you CTRL+C it prematurely! I don't know why I haven't thought about that before...
Download |██████████████████⚠︎ | (!) 45/100 [45%] in 4.8s (9.43/s)
And finally, you can choose to disable CTRL+C at all! The default is the safer ctrl_c=True
, which does make CTRL-C work as usual.
Disable it ctrl_c=False
, to make your interactive alive_bar
much smoother to use (there are no stack traces if you stop it), and/or if it is at the top-level of your program!
Beware: If it is e.g. inside a for-loop, it will just continue to the next iteration, which may or may not be what you want...
for i in range(10):
with alive_bar(100, ctrl_c=False, title=f'Download {i}') as bar:
for i in range(100):
time.sleep(0.02)
bar()
Download 0 |████████▊⚠︎ | (!) 22/100 [22%] in 0.6s (36.40/s)
Download 1 |████████████████▊⚠︎ | (!) 42/100 [42%] in 1.0s (41.43/s)
Download 2 |██████▍⚠︎ | (!) 16/100 [16%] in 0.4s (39.29/s)
Download 3 |█████▋⚠︎ | (!) 14/100 [14%] in 0.4s (33.68/s)
Download 4 |█████████████▎⚠︎ | (!) 33/100 [33%] in 0.8s (39.48/s)
Download 5 |███████▎⚠︎ | (!) 18/100 [18%] in 0.5s (37.69/s)
Download 6 |█████▎⚠︎ | (!) 13/100 [13%] in 0.3s (37.28/s)
Download 7 |████████████⚠︎ | (!) 30/100 [30%] in 0.8s (38.43/s)
Download 8 |██████⚠︎ | (!) 15/100 [15%] in 0.4s (36.26/s)
...
Some major new features, often requested, have finally landed!
click.echo()
printingYES! Now alive-progress
has support for Jupyter Notebooks and also includes a Disabled state! Both were highly sought after, and have finally landed!
And better, I've implemented an auto-detection mechanism for jupyter notebooks, so it just works, out of the box, without any changes in your code!!
See for yourself:
It seems to work very well, but at this moment, it should be considered experimental.
There were instances in which some visual glitches have appeared, like twoalive_bar
refreshes being concatenated together instead of over one another... And it's something I think I can't possibly work around: it seems Jupyter sometimes refresh the canvas at odd times, which makes it lose some data. Please let me know on the issues if something funnier arises.
This is a major breakthrough in alive-progress
!
I took 1 year developing it, and I'm very proud of what I've accomplished \o/
.check()
tools that compile and beautifully render all frames from all animation cycles of spinners and bars! they can even include complete frame data, internal codepoints, and even their animations! 👏alive-progress
widgets!alive_it
, that accepts an iterable and calls bar()
for you!Since this is a major version change, direct backward compatibility is not guaranteed. If something does not work at first, just check the new imports and functions' signatures, and you should be good to go. All previous features should still work here! 👍
alive-progress
Just install with pip:
❯ pip install alive-progress
If you're wondering what styles are builtin, it's showtime
! ;)
from alive_progress.styles import showtime
showtime()
Note: Please disregard the path in the animated gif below, the correct one is above. These long gifs are very time-consuming to generate, so I can't make another on every single change. Thanks for your understanding.
I've made these styles just to try all the animation factories I've created, but I think some of them ended up very, very cool! Use them at will, and mix them to your heart's content!
Do you want to see actual alive-progress
bars gloriously running in your system before trying them yourself?
❯ python -m alive_progress.tools.demo
Cool, huh?? Now enter an ipython
REPL and try this:
from alive_progress import alive_bar
import time
for x in 1000, 1500, 700, 0:
with alive_bar(x) as bar:
for i in range(1000):
time.sleep(.005)
bar()
You'll see something like this, with cool animations throughout the process 😜:
|████████████████████████████████████████| 1000/1000 [100%] in 5.8s (171.62/s)
|██████████████████████████▋⚠︎ | (!) 1000/1500 [67%] in 5.8s (172.62/s)
|████████████████████████████████████████✗︎ (!) 1000/700 [143%] in 5.8s (172.06/s)
|████████████████████████████████████████| 1000 in 5.8s (172.45/s)
Nice, huh? Loved it? I knew you would, thank you 😊.
To actually use it, just wrap your normal loop in an alive_bar
context manager like this:
with alive_bar(total) as bar: # declare your expected total
for item in items: # <<-- your original loop
print(item) # process each item
bar() # call `bar()` at the end
And it's alive! 👏
So, in short: retrieve the items as always, enter the alive_bar
context manager with the number of items, and then iterate/process those items, calling bar()
at the end! It's that simple! :)
items
can be any iterable, like for example, a queryset;alive_bar
is the expected total, like qs.count()
for querysets, len(items)
for iterables with length, or even a static number;bar()
is what makes the bar go forward — you usually call it in every iteration, just after finishing an item;bar()
too much (or too few at the end), the bar will graphically render that deviation from the expected total
, making it very easy to notice overflows and underflows;bar.current
.You can get creative! Since the bar only goes forward when you call
bar()
, it is independent of the loop! So you can use it to monitor anything you want, like pending transactions, broken items, etc., or even call it more than once in the same iteration! So, in the end, you'll get to know how many of those "special" events there were, including their percentage relative to the total!
While inside an alive_bar
context, you can effortlessly display messages tightly integrated with the current progress bar being displayed! It won't break in any way and will even enrich your message!
bar.text('message')
and bar.text = 'message'
set a situational message right within the bar, where you can display something about the current item or the phase the processing is in;bar.title('Title')
and bar.title = 'Title'
— mix with title_length
to keep the bar from changing its length;print()
statement, where alive_bar
nicely cleans up the line, prints your message alongside the current bar position at the time, and continues the bar right below it;logging
framework, including file outputs, is also enriched exactly like the previous one;click.echo()
to print styled text.Awesome right? And all of these work just the same in a terminal or in a Jupyter notebook!
You now have a quicker way to monitor anything! Here, the items are automatically tracked for you!
Behold the alive_it
=> the alive_bar
iterator adapter!
Simply wrap your items with it, and loop over them as usual!
The bar will just work; it's that simple!
from alive_progress import alive_it
for item in alive_it(items): # <<-- wrapped items
print(item) # process each item
HOW COOL IS THAT?! 😜
All alive_bar
parameters apply but total
, which is smarter (if not supplied, it will be auto-inferred from your data using len
or length_hint
), and manual
that does not make sense here.
Note there isn't any bar
handle at all in there. But what if you do want it, e.g. to set text messages or retrieve the current progress?
You can interact with the internal alive_bar
by just assigning alive_it
to a variable like this:
bar = alive_it(items) # <<-- bar with wrapped items
for item in bar: # <<-- iterate on bar
print(item) # process each item
bar.text(f'ok: {item}') # WOW, it works!
Note that this is a slightly special bar
, which does not support bar()
, since the iterator adapter tracks items automatically for you. Also, it supports finalize
, which enables you to set the title and/or text of the final receipt:
alive_it(items, finalize=lambda bar: bar.text('Success!'))
...
In a nutshell:
- full use is always
with alive_bar() as bar
, where you iterate and callbar()
whenever you want;- quick adapter use is
for item in alive_it(items)
, where items are automatically tracked;- full adapter use is
bar = alive_it(items)
, where in addition to items being automatically tracked, you get a special iterablebar
able to customize the inneralive_progress
however you want.
Actually, the total
argument is optional. If you do provide it, the bar enters in definite mode, the one used for well-bounded tasks. This mode has all the widgets alive-progress
has to offer: progress, count, throughput, and ETA.
If you don't, the bar enters in unknown mode, the one used for unbounded tasks. In this mode, the whole progress bar is animated, as it's not possible to determine the progress, and therefore the ETA. But you still get the count and throughput widgets as usual.
The cool spinner is still present here alongside the progress bar, both running their animations concurrently and independently of each other, rendering a unique show in your terminal! 😜
Both definite and unknown modes use internally a counter to maintain progress. This is the source value which all widgets are derived from.
On the other hand, the manual mode uses internally a percentage to maintain progress. This enables you to get complete control of the bar position! It's usually used to monitor processes that only feed you the percentage of completion, or to generate some kind of special effects.
To use it, just include a manual=True
argument into alive_bar
(or config_handler
), and you get to send any percentage to the bar()
handler! For example, to set it to 15%, just call bar(0.15)
— which is 15 / 100.
You can also use total
here! If you do provide it, alive-progress
will infer an internal counter by itself, and thus will be able to offer you the same count, throughput, and ETA widgets!
If you don't, you'll at least get rough versions of the throughput and ETA widgets. The throughput will use "%/s" (percent per second), and the ETA will be till 1.0 (100%). Both are very inaccurate but better than nothing.
You can call bar
in manual mode as frequently as you want! The refresh rate will still be asynchronously computed as usual, according to the current progress and the elapsed time, so you won't ever spam the terminal with more updates than it can handle.
When total
is provided all is cool:
mode | counter | percentage | throughput | ETA | over/underflow |
---|---|---|---|---|---|
definite | ✅ (user tick) | ✅ (inferred) | ✅ | ✅ | ✅ |
manual | ✅ (inferred) | ✅ (user set) | ✅ | ✅ | ✅ |
When it isn't, some compromises have to be made:
mode | counter | percentage | throughput | ETA | over/underflow |
---|---|---|---|---|---|
unknown | ✅ (user tick) | ❌ | ✅ | ❌ | ❌ |
manual | ❌ | ✅ (user set) | ⚠️ (simpler) | ⚠️ (rough) | ✅ |
But actually it's quite simple, you do not need to think about which mode you should use:
Just always send the total
if you have it, and use manual
if you need it!
It will just work the best it can! 👏 \o/
bar()
handlersThe bar()
handlers support either relative or absolute semantics, depending on the mode:
bar()
to increment the counter by one, or send any other positive increment like bar(5)
to increment by those at once;bar(0.35)
to instantly put the bar in 35% position — this argument is mandatory here!The manual modes enable you to get super creative! Since you can set the bar instantly to whatever position you want, you could:
- make it go backwards — perhaps to graphically display the timeout of something;
- create special effects — perhaps to act as a real-time gauge of some sort.
In any case, to retrieve the current count/percentage, just call: bar.current
:
Last but not least, the bar()
handler of the definite mode has a unique ability: skipping items for an accurate ETA! Just call bar(skipped=False)
or bar(skipped=True)
to use it. When skipped is True, that item(s) are ignored when computing the rate, and thus not ruining the ETA.
Maintaining an open source project is hard and time-consuming, and I've put much ❤️ and effort into this.
If you've appreciated my work, you can back me up with a donation! Thank you 😊
The showtime
exhibit has an optional argument to choose which show to present, Show.SPINNERS
(default), Show.BARS
or Show.THEMES
, do take a look at them! ;)
from alive_progress.styles import showtime, Show
Note: Please disregard the path in the animated gif below, the correct one is above. These long gifs are very time-consuming to generate, so I can't make another on every single change. Thanks for your understanding.
And the themes one (📌 new in 2.0):
The showtime
exhibit also accepts some customization options:
For example to get a marine show, you can showtime(pattern='boat|fish|crab')
:
You can also access these shows with the shorthands
show_bars()
,show_spinners()
, andshow_themes()
!
There's also a small utility called
print_chars()
, to help find that cool character to put in your customized spinners and bars, or to determine if your terminal does support Unicode characters.
There are several options to customize both appearance and behavior!
All of them can be set both directly in the alive_bar
or globally in the config_handler
!
These are the options - default values in brackets:
title
: an optional, always visible bar titlelength
: [40
] the number of cols to render the animated progress barmax_cols
: [80
] the maximum cols to use if not possible to fetch it, like in jupyterspinner
: the spinner style to be rendered next to the bar
bar
: the bar style to be rendered in known modes
unknown
: the bar style to be rendered in the unknown mode
theme
: ['smooth'
] a set of matching spinner, bar, and unknown
force_tty
: [None
] forces animations to be on, off, or according to the tty (more details here)
file
: [sys.stdout
] the file object to use: sys.stdout
, sys.stderr
, or a similar TextIOWrapper
disable
: [False
] if True, completely disables all output, do not install hooksmanual
: [False
] set to manually control the bar positionenrich_print
: [True
] enriches print() and logging messages with the bar positionreceipt
: [True
] prints the nice final receipt, disables if Falsereceipt_text
: [False
] set to repeat the last text message in the final receiptmonitor
(bool|str): [True
] configures the monitor widget 152/200 [76%]
{count}
, {total}
and {percent}
to customize itelapsed
(bool|str): [True
] configures the elapsed time widget in 12s
{elapsed}
to customize itstats
(bool|str): [True
] configures the stats widget (123.4/s, eta: 12s)
{rate}
and {eta}
to customize itmonitor_end
(bool|str): [True
] configures the monitor widget within final receipt
monitor
's oneelapsed_end
(bool|str): [True
] configures the elapsed time widget within final receipt
elapsed
's onestats_end
(bool|str): [True
] configures the stats widget within final receipt
{rate}
to customize it (no relation to stats)title_length
: [0
] fixes the length of titles, or 0 for unlimited
spinner_length
: [0
] forces the spinner length, or 0
for its natural onerefresh_secs
: [0
] forces the refresh period to this, 0
is the reactive visual feedbackctrl_c
: [True
] if False, disables CTRL+C (captures it)dual_line
: [False
] if True, places the text below the barunit
: any text that labels your entitiesscale
: the scaling to apply to units: None
, SI
, IEC
, or SI2
False
or ''
-> None
, True
-> SI
, 10
or '10'
-> SI
, 2
or '2'
-> IEC
precision
: [1
] how many decimals do display when scalingAnd there's also one that can only be set locally in an alive_bar
context:
calibrate
: maximum theoretical throughput to calibrate the animation speed (more details here)To set them locally, just send them as keyword arguments to alive_bar
:
with alive_bar(total, title='Processing', length=20, bar='halloween') as bar:
...
To use them globally, send them to config_handler
, and any alive_bar
created after that will include those options! And you can mix and match them, local options always have precedence over global ones:
from alive_progress import config_handler
config_handler.set_global(length=20, spinner='wait')
with alive_bar(total, bar='blocks', spinner='twirls') as bar:
# the length is 20, the bar is 'blocks' and the spinner is 'twirls'.
...
Yes, you can assemble your own spinners! And it's easy!
I've created a plethora of special effects, so you can just mix and match them any way you want! There are frames, scrolling, bouncing, sequential, alongside, and delayed spinners! Get creative! 😍
The spinners' animations are engineered by very advanced generator expressions, deep within several layers of meta factories, factories and generators 🤯!
alive_bar
and config_handler
;These generators are capable of multiple different animation cycles according to the spinner behavior, e.g. a bouncing spinner can run one cycle to smoothly bring a subject into the scene, then repeatedly reposition it until the other side, then make it smoothly disappear off the scene => and this is all only one cycle! Then it can be followed by another cycle to make it all again but backwards! And bouncing spinners also accept different and alternating patterns in both the right and left directions, which makes them generate the cartesian product of all the combinations, possibly producing dozens of different cycles until they start repeating them!! 🤯
And there's more, I think one of the most impressive achievements I got in this animation system (besides the spinner compiler itself)... They only yield more animation frames until the current cycle is not exhausted, then they halt themselves! Yep, the next cycle does not start just yet! This behavior creates natural breaks in exactly the correct spots, where the animations are not disrupted, so I can smoothly link with whatever other animation I want!!
This has all kinds of cool implications: the cycles can have different frame counts, different screen lengths, they do not need to be synchronized, they can create long different sequences by themselves, they can cooperate to play cycles in sequence or alongside, and I can amaze you displaying several totally distinct animations at the same time without any interferences whatsoever!
It's almost like they were... alive!! 😄
==> Yes, that's where this project's name came from! 😉
Now, these generators of cycles and frames are fully consumed ahead of time by the Spinner Compiler! This is a very cool new processor that I made inside the Cell Architecture effort, to make all these animations work even in the presence of wide chars or complex grapheme clusters! It was very hard to make these clusters gradually enter and exit frames, smoothly, while keeping them from breaking the Unicode encoding and especially maintain their original lengths in all frames! Yes, several chars in sequence can represent another completely different symbol, so they cannot ever be split! They have to enter and exit the frame always together, all at once, or the grapheme won't show up at all (an Emoji for instance)!! Enter the Spinner Compiler......
This has made possible some incredible things!! Since this Compiler generates the whole spinner frame data beforehand:
Also, with the complete frame data compiled and persisted, I could create several commands to refactor that data, like changing shapes, replacing chars, adding visual pauses (frame repetitions), generating bouncing effects on-demand over any content, and even transposing cycles with frames!!
But how can you see these effects? Does the effect you created look good? Or is it not working as you thought? YES, now you can see all generated cycles and frames analytically, in a very beautiful rendition!!
I love what I've achieved here 😊, it's probably THE most beautiful tool I've ever created... Behold the check
tool!!
It's awesome if I say so myself, isn't it? And a very complex piece of software I'm proud of, take a look at its code if you'd like.
And the check
tool is much more powerful! For instance, you can see the codepoints of the frames!!! And maybe have a glimpse of why this version was so, so very hard and complex to make...
In red, you see the grapheme clusters, that occupy one or two "logical positions", regardless of their actual sizes... These are the "Cells" of the new Cell Architecture...
Look how awesome an Emoji Flag is represented:
The flag seems to move so smoothly because it uses "half-characters"! Since it is a wide char, alive-progress
knows it will be rendered with "two visible chars", and the animations consider this, but compose with spaces, which occupy only one. When one uses mixed backgrounds, the situation is much more complex...
The types of factories I've created are:
frames
: draws any sequence of characters at will, that will be played frame by frame in sequence;scrolling
: generates a smooth flow from one side to the other, hiding behind or wrapping upon invisible borders — allows using subjects one at a time, generating several cycles of distinct characters;bouncing
: similar to scrolling
, but makes the animations bounce back to the start, hiding behind or immediately bouncing upon invisible borders;sequential
get a handful of factories and play them one after the other sequentially! allows to intermix them or not;alongside
get a handful of factories and play them alongside simultaneously, why choose when you can have them all?! allows to choose the pivot of the animation;delayed
: get any other factory and copy it multiple times, increasingly skipping some frames on each one! very cool effects are made here!For more details please look at their docstrings, which are very complete.
Customizing bars is nowhere near that involved. Let's say they are "immediate", passive objects. They do not support animations, i.e. they will always generate the same rendition given the same parameters. Remember spinners are infinite generators, capable of generating long and complex sequences.
Well, bars also have a meta factory, use closures to store the styling parameters, and receive additional operating parameters, but then the actual factory can't generate any content by itself. It still needs an extra parameter, a floating-point number between 0 and 1, which is the percentage to render itself.
alive_bar
calculates this percentage automatically based on the counter and total, but you can send it yourself when in themanual
mode!
Bars also do not have a Bar Compiler, but they do provide the check tool!! 🎉
You can even mix and match wide chars and normal chars just like in spinners! (and everything keeps perfectly aligned 😅)
Use the check tools to your heart's content!! They have even more goodies awaiting you, even real-time animations!
Create the wildest and coolest animations you can and send them to me!
I'm thinking about creating some kind ofcontrib
package, with user-contributed spinners and bars!
Wow, if you've read everything till here, you should now have a sound knowledge about using alive-progress
! 👏
But brace yourself because there is even more, exciting stuff lies ahead!
Maintaining an open source project is hard and time-consuming, and I've put much ❤️ and effort into this.
If you've appreciated my work, you can back me up with a donation! Thank you 😊
Oh, you want to pause it altogether, I hear? This is an amazing novel concept, not found anywhere AFAIK.
With this you get to act on some items manually, at will, right in the middle of an ongoing processing!!
YES, you can return to the prompt and fix, change, submit things, and the bar will just "remember" where it was...
Suppose you need to reconcile payment transactions (been there, done that). You need to iterate over thousands of them, detect somehow the faulty ones, and fix them. This fix is not simple nor deterministic, you need to study each one to understand what to do. They could be missing a recipient, or have the wrong amount, or not be synced with the server, etc., it's hard to even imagine all possibilities.
Typically, you would have to let the detection process run until completion, appending to a list each inconsistency it finds and waiting, potentially a long time, until you can finally start fixing them... You could of course mitigate that by processing in chunks, or printing them and acting via another shell, etc., but those have their own shortcomings... 😓
Now, there's a better way! Simply pause the actual detection process for a while! Then you just have to wait till the next fault is found, and act in near real-time!
To use the pause mechanism you just have to write a function, so the code can yield
the items you want to interact with. You most probably already use one in your code, but in the ipython
shell or another REPL you probably don't. So just wrap your debug code in a function, then enter within a bar.pause()
context!!
def reconcile_transactions():
qs = Transaction.objects.filter() # django example, or in sqlalchemy: session.query(Transaction).filter()
with alive_bar(qs.count()) as bar:
for transaction in qs:
if faulty(transaction):
with bar.pause():
yield transaction
bar()
That's it! It's that simple! \o/
Now run gen = reconcile_transactions()
to instantiate the generator, and whenever you want the next faulty transaction, just call next(gen, None)
! I love it...
The alive-progress
bar will start and run as usual, but as soon as any inconsistency is found, the bar will pause itself, turning off the refresh thread and remembering its exact state, and yield the transaction to you directly on the prompt! It's almost magic! 😃
In [11]: gen = reconcile_transactions()
In [12]: next(gen, None)
|█████████████████████ | 105/200 [52%] in 5s (18.8/s, eta: 4s)
Out[12]: Transaction<#123>
You can then inspect the transaction with the usual _
shortcut of ipython
(or just directly assign it with t = next(gen, None)
), and you're all set to fix it!
When you're done, just reactivate the bar with the same next
call as before!! The bar reappears, turns everything back on, and continues like it had never stopped!! Ok, it is magic 😜
In [21]: next(gen, None)
|█████████████████████ | ▁▃▅ 106/200 [52%] in 5s (18.8/s, eta: 4s)
Rinse and repeat till the final receipt appears, and there'll be no faulty transactions anymore. 😄
So, you need to monitor a fixed operation, without any loops, right?
It'll work for sure! Here is a naive example (we'll do better in a moment):
with alive_bar(4) as bar:
corpus = read_file(file)
bar() # file was read, tokenizing
tokens = tokenize(corpus)
bar() # tokens generated, processing
data = process(tokens)
bar() # process finished, sending response
resp = send(data)
bar() # we're done! four bar calls with `total=4`
It's naive because it assumes all steps take the same amount of time, but actually, each one may take a very different time to complete. Think read_file
and tokenize
may be extremely fast, which makes the percentage skyrocket to 50%, then stopping for a long time in the process
step... You get the point, it can ruin the user experience and create a very misleading ETA.
To improve upon that you need to distribute the steps' percentages accordingly! Since you told alive_bar
there were four steps, when the first one was completed it understood 1/4 or 25% of the whole processing was complete... Thus, you need to measure how long your steps actually take and use the manual mode to increase the bar percentage by the right amount at each step!
You can use my other open source project about-time to easily measure these durations! Just try to simulate with some representative inputs, to get better results. Something like:
from about_time import about_time
with about_time() as t_total: # this about_time will measure the whole time of the block.
with about_time() as t1 # the other four will get the relative timings within the whole.
corpus = read_file(file) # `about_time` supports several calling conventions, including one-liners.
with about_time() as t2 # see its documentation for more details.
tokens = tokenize(corpus)
with about_time() as t3
data = process(tokens)
with about_time() as t4
resp = send(data)
print(f'percentage1 = {t1.duration / t_total.duration}')
print(f'percentage2 = {t2.duration / t_total.duration}')
print(f'percentage3 = {t3.duration / t_total.duration}')
print(f'percentage4 = {t4.duration / t_total.duration}')
There you go! Now you know the relative timings of all the steps, and can use them to improve your original code! Just get the cumulative timings and put them within a manual mode alive_bar
!
For example, if the timings you found were 10%, 30%, 20%, and 40%, you'd use 0.1, 0.4, 0.6, and 1.0 (the last one should always be 1.0):
with alive_bar(4, manual=True) as bar:
corpus = read_big_file()
bar(0.1) # 10%
tokens = tokenize(corpus)
bar(0.4) # 30% + 10% from previous steps
data = process(tokens)
bar(0.6) # 20% + 40% from previous steps
resp = send(data)
bar(1.) # always 1. in the last step
That's it! The user experience and ETA should be greatly improved now.
So, you want to calibrate the engine?
The alive-progress
bars have cool visual feedback of the current throughput, so you can actually see how fast your processing is, as the spinner runs faster or slower with it.
For this to happen, I've put together and implemented a few fps curves to empirically find which one gave the best feel of speed:
(interactive version [here](https://www.desmos.com/calculator/ema05elsux))
The graph shows the logarithmic (red), parabolic (blue) and linear (green) curves, these are the ones I started with. It was not an easy task, I've made dozens of tests, and never found one that really inspired that feel of speed I was looking for. The best one seemed to be the logarithmic one, but it reacted poorly with small numbers. I know I could make it work with a few twists for those small numbers, so I experimented a lot and adjusted the logarithmic curve (dotted orange) until I finally found the behavior I expected! It is the one that seemed to provide the best all-around perceived speed changes throughout the whole spectrum from a few to billions... That is the curve I've settled with, and it's the one used in all modes and conditions. In the future and if someone would find it useful, that curve could be configurable.
Well, the default alive-progress
calibration is 1,000,000 in bounded modes, i.e., it takes 1 million iterations per second for the bar to refresh itself at 60 frames per second. In the manual unbounded mode, it is 1.0 (100%). Both enable a vast operating range and generally work quite well.
For example, take a look at the effect these very different calibrations have, running the very same code at the very same speed! Notice the feel the spinner passes to the user, is this processing going slow or going fast? And remember that isn't only the spinner refreshing but the whole line, complete with the bar rendition and all widgets, so everything gets smoother or sluggish:
So, if your processing hardly gets to 20 items per second, and you think
alive-progress
is rendering sluggish, you could increase that sense of speed by calibrating it to let's say40
, and it will be running waaaay faster... It is better to always leave some headroom and calibrate it to something between 50% and 100% more, and then tweak it from there to find the one you like the most! :)
Do these astonishing alive-progress
animations refuse to display?
PyCharm is awesome, I love it! But I'll never understand why they've disabled emulating a terminal by default... If you do use PyCharm's output console, please enable this on all your Run Configurations:
I even recommend you go into
File
>New Projects Setup
>Run Configuration Templates
, selectPython
, and also enable it there, so any new ones you create will already have this set.
In addition to that, some terminals report themselves as "non-interactive", like when running out of a real terminal (PyCharm and Jupyter for example), in shell pipelines (cat file.txt | python program.py
), or in background processes (not connected to a tty).
When alive-progress
finds itself in a non-interactive terminal, it automatically disables all kinds of animations, printing only the final receipt. This is made in order to avoid both messing up the pipeline output and spamming your log file with thousands of alive-progress
refreshes.
So, when you know it's safe, you can force them to see alive-progress
in all its glory! Here is the force_tty
argument:
with alive_bar(1000, force_tty=True) as bar:
for i in range(1000):
time.sleep(.01)
bar()
The values accepted are:
force_tty=True
-> always enables animations, and auto-detects Jupyter Notebooks!force_tty=False
-> always disables animations, keeping only the final receiptforce_tty=None
(default) -> auto detect, according to the terminal's tty stateYou can also set it system-wide using config_handler
, so you don't need to pass it manually anymore.
Do note that PyCharm's console and Jupyter notebooks are heavily instrumented and thus have much more overhead, so the outcome may not be as fluid as you would expect. On top of that, Jupyter notebooks do not support ANSI Escape Codes, so I had to develop some workarounds to emulate functions like "clear the line" and "clear from cursor"... To see the fluid and smooth
alive_bar
animations as I intended, always prefer a full-fledged terminal.
alive-progress
hadn't had any dependency. Now it has two: one is about-time (another very cool project of mine if I say so myself), to track the spinner compilation times and generate its human-friendly renditions. The other is grapheme, to detect grapheme cluster breaks (I've opened an issue there asking about the future and correctness of it, and the author guarantees he intends to update the project on every new Unicode version);alive-progress
hadn't had a single Python class! Now it has a few tiny ones for very specific reasons (change callables, iterator adapter, and some descriptors for the widgets).
alive_bar
itself is a function! And in the latter mostly, I dynamically plug several other functions into the main one (Python functions have a __dict__
just like classes do). 😝alive_bar
does notice terminal size changes, but just truncates the line accordingly)contrib
system somehow, to allow a simple way to share cool spinners and bars from usersskipped
itemsstderr
and other files instead of stdout
monitor
, elapsed
, stats
Complete here.
alive_it
calls, detect nested uses of alive_progress and throw a clearer error messagealive_it
skipped
items, new max_cols
config setting for jupyter, fix fetching the size of the terminal when using stderr, officially supports Python 3.11sys.stderr
and other files instead of sys.stdout
, smoothed out the rate estimation, more queries into the currently running widgets' data, help system in configuration errorsmonitor
, elapsed
, and stats
core widgets, new monitor_end
, elapsed_end
, and stats_end
core widgets, better support for CTRL+C, which makes alive_bar
stop prematurelyclick.echo()
support; faster performance; safer detection of terminal columns; bar.current
acts like a property; remove Python 3.6.check()
tools in both spinners and bars; bars and spinners engines revamp; new animation modes in alongside and sequential spinners; new builtin spinners, bars, and themes; dynamic showtime with themes, scroll protection and filter patterns; improved logging for files; several new configuration options for customizing appearance; new iterator adapter alive_it
; uses time.perf_counter()
high-resolution clock; requires Python 3.6+ (and officially supports Python 3.9 and 3.10)bar.current()
method; newlines get printed on vanilla Python REPL; the bar is truncated to 80 chars on Windowsbar.text()
method, to set situational messages at any time, without incrementing position (deprecates 'text' parameter in bar()
); performance optimizationsbackground
parameter instead of blank
, which accepts arbitrarily sized strings and remains fixed in the background, simulating a bar going "over it"show_spinners
and show_bars
, new utility print_chars
, show_bars
gain some advanced demonstrations (try it again!)alive_progress
will always try to keep up with Python, so starting from version 2.0, I'll drop support for all Python versions which enter EoL. See their schedule here.
But don't worry if you can't migrate just yet: alive_progress
versions are perennial, so just keep using the one that works for you and you're good.
I just strongly recommend setting older alive_progress
packages in a requirements.txt file with the following formats. These will always fetch the latest build releases previous to a given version, so, if I ever release bug fixes, you'll get them too.
❯ pip install -U "alive_progress<2"
❯ pip install -U "alive_progress<2.2"
This software is licensed under the MIT License. See the LICENSE file in the top distribution directory for the full license text.
Maintaining an open source project is hard and time-consuming, and I've put much ❤️ and effort into this.
If you've appreciated my work, you can back me up with a donation! Thank you 😊