Friday, April 21, 2017

Python callbacks using classes and methods

By Vasudev Ram

Hi readers,

I had written this post a few days ago:

Implementing and using callbacks in Python

In it, I had shown how to create simple callbacks using just plain Python functions, and said I would write a bit more about callbacks in my next post.

This is that next post. It discusses how to create callbacks in Python using classes and methods.

Here is an example program, callback_demo2.py, that shows how to do that:
'''
File: callback_demo2.py
To demonstrate implementation and use of callbacks in Python, 
using classes with methods as the callbacks.
Author: Vasudev Ram
Copyright 2017 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
'''

from __future__ import print_function
import sys
from time import sleep

class FileReporter(object):

    def __init__(self, filename):
        self._filename = filename
        try:
            self._fil = open(self._filename, "w")
        except IOError as ioe:
            sys.stderr.write("While opening {}, caught IOError: {}\n".format(
                self._filename, repr(ioe)))
            sys.exit(1)

    def report(self, message):
        self._fil.write(message)

class ScreenReporter(object):

    def __init__(self, dest):
        self._dest = dest

    def report(self, message):
        self._dest.write(message)

def square(i):
    return i * i

def cube(i):
    return i * i * i

def processor(process, times, report_interval, reporter):
    result = 0
    for i in range(1, times + 1):
        result += process(i)
        sleep(0.1)
        if i % report_interval == 0:
            # This is the call to the callback method 
            # that was passed to this function.
            reporter.report("Items processed: {}. Running result: {}.\n".format(i, result))

file_reporter = FileReporter("processor_report.txt")
processor(square, 20, 5, file_reporter)

stdout_reporter = ScreenReporter(sys.stdout)
processor(square, 20, 5, stdout_reporter)

stderr_reporter = ScreenReporter(sys.stderr)
processor(square, 20, 5, stderr_reporter)
I ran it with:
$ python callback_demo2.py >out 2>err
The above command creates 3 files, processor_report.txt, out and err.
Running fc /l on those 3 files, pairwise, shows that all three have the same content, but the output has gone to 3 different destinations (a specified file, standard output, and standard error output, based on which callback was passed to the processor function, in the 3 calls to it.

These two lines from the program:
file_reporter = FileReporter("processor_report.txt")
processor(square, 20, 5, file_reporter)
send output to the file processor_report.txt.

These two lines:
stdout_reporter = ScreenReporter(sys.stdout)
processor(square, 20, 5, stdout_reporter)
send output to the standard output (stdout), which is redirected to the file out.

These two lines:
stderr_reporter = ScreenReporter(sys.stderr)
processor(square, 20, 5, stderr_reporter)
send output to the standard error output (stderr), which is redirected to the file err.

The difference between this program (callback_demo2.py) and the one in the previous post (callback_demo.py), is that in this one, I pass an instance of some class to the processor function, as its last argument. This argument is the callback. And this time, rather than treating it as a function, processor treats it as an object, and invokes the report method on it, giving us much the same output as before (I just made minor cosmetic changes in the output). This same thing is done in all the 3 calls to processor. The difference is that different types of objects are passed each time, for the callback argument.

[ An interesting side note here is that in some other languages, for example, Java, at least in earlier Java versions (I'm not sure about recent ones), we cannot just pass different types of objects for the same callback parameter, unless they are all derived from some common base class (via inheritance). (Can it be done using Java interfaces? Need to check - been a while.) While that is also possible in Python, it is not necessary, as we see here, due to Python's more dynamic nature. Just passing any object that implements a report method is enough, as the program shows. The FileReporter and ScreenReporter classes do not (need to) descend from some common base class (other than object, but that is just the syntax for new-style classes; the object class not provide the report method). Python's smooth polymorphism, a.k.a. duck typing, takes care of it. ]

The first time, a FileReporter object is passed, so the output goes to a text file.

The second and third times, a ScreenReporter object is passed, initializing the _dest field to stdout and stderr respectively, so the output goes to those two destinations respectively. And the command line redirects those outputs to the files out and err.

Although I didn't say so in the previous post, the first argument to processor (square), is also a callback, since it, in turn, is called from processor, via the process parameter. So I can pass some other argument in place of it, like the cube function defined in the program, to get different computations done and hence different results.




Keen on creating and selling products online? Check out the free Product Creation Masterclass. It runs for about 4 weeks from 24 April 2017 (that's 3 days from now), with lots of emails, videos and expert interviews, all geared toward helping you create and sell your first product online. Check it out: The Product Creation Masterclass.

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Are you a blogger with some traffic? Get Convertkit:

Email marketing for professional bloggers



Tuesday, April 18, 2017

Implementing and using callbacks in Python

By Vasudev Ram

Callbacks (or callback functions, as they are also called) are a useful and somewhat powerful programming technique. I first learned about the technique (in C) during a course on X-Windows , XLib and Motif, that I attended years ago.

Wikipedia article about callbacks.

( Also see this :)

In this post I'll show a simple way of implementing callbacks in Python. Due to Python's dynamic nature, it is quite easy to implement callbacks in it, more so than in some other languages, like C++ or Java. (But it's not difficult in those languages either.)

The program callback_demo.py (below) shows a simple way to implement and use callbacks, using just plain functions, i.e. no classes or anything more sophisticated are needed:
# File: callback_demo.py
# To demonstrate implementation and use of callbacks in Python, 
# using just plain functions.
# Author: Vasudev Ram
# Copyright 2017 Vasudev Ram
# Web site: https://vasudevram.github.io
# Blog: https://jugad2.blogspot.com
# Product store: https://gumroad.com/vasudevram

from __future__ import print_function
from time import sleep

def callback_a(i, result):
    print("Items processed: {}. Running result: {}.".format(i, result))

def square(i):
    return i * i

def processor(process, times, report_interval, callback):
    print("Entered processor(): times = {}, report_interval = {}, callback = {}".format(
    times, report_interval, callback.func_name))
    # Can also use callback.__name__ instead of callback.func_name in line above.
    result = 0
    print("Processing data ...")
    for i in range(1, times + 1):
        result += process(i)
        sleep(1)
        if i % report_interval == 0:
            # This is the call to the callback function 
            # that was passed to this function.
            callback(i, result)

processor(square, 20, 5, callback_a)
And here is the output when I run it:
$ python callback_demo.py
Entered processor(): times = 20, report_interval = 5, callback = callback_a
Processing data ...
Items processed: 5. Running result: 55.
Items processed: 10. Running result: 385.
Items processed: 15. Running result: 1240.
Items processed: 20. Running result: 2870.
$

The function callback_a (the last argument to the call to processor), gets substituted for the function parameter callback in the processor function definition. So callback_a gets called, and it reports the progress of the work being done. If we passed a different callback function instead of callback_a, we could get different behavior, e.g. progress report in a different format, or to a different destination, or something else, without changing the actual processor function. So this is a way of creating functions with low coupling, or putting it in other terms, creating functions (like processor) that have high cohesion.

Read up about coupling and cohesion.

Note that in the program, callback_demo.py, I've shown two ways of getting the name of a function, one being callback.func_name and the other being callback.__name__ (the latter in a comment). Both ways work. Also see this code snippet, which shows that if you define a function foo, it's func_name attribute is 'foo'; if you then assign foo to bar, bar's func_name attribute is still 'foo', not 'bar':
>>> def foo(): pass
...
>>> foo.__name__
'foo'
>>> bar = foo
>>>
>>> bar.__name__
'foo'
>>>
>>> foo.func_name
'foo'
>>> bar.func_name
'foo'
I'll talk a bit more about callbacks in my next post.

Enjoy.

- Vasudev Ram - Online Python training and consulting

Do you have the yen to create products? Check out the Product Creation Masterclass. (Not yen as in cash - the class is free.) It runs for about 4 weeks from 24 April 2017, with lots of emails and videos and interviews, all geared toward helping you create and sell your first product online. Check it out: The Product Creation Masterclass.

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Are you a blogger with some traffic? Get Convertkit:

Email marketing for professional bloggers



Saturday, April 15, 2017

Andrei Alexandrescu's AMA on IAmA reddit (C++ and D are some topics)

By Vasudev Ram

Just saw this today via a link while browsing articles about DLang (the D Language):

Andrei Alexandrescu's AMA on IAmA reddit

It starts off like this:

[ I am a member of Facebook's HHVM team, a C++ and D pundit, and a Machine Learning guy. Ask me anything!

Background:
My name is Andrei Alexandrescu. I've been a Research Scientist at Facebook for almost four years, during which I've worked on various projects mainly around Machine Learning and systems programming. In January I joined the HHVM project, which aims at improving the productivity and performance of PHP on Facebook's server infrastructure.
I've also written a couple of books: Modern C++ Design, C++ Coding Standards (together with Herb Sutter), and The D Programming Language, as well as some papers and articles. Some of my talks are on video (this is the most recent).
Looking forward to answering your questions! ]

I read part of the thread (so far); there are some interesting questions and answers in it, about both C++ and D (and maybe some other stuff I haven't seen yet). One such interesting point was this, where, in reply to a question, he says he likes to brag about inventing D's scope statement :-)

Some excerpts below:

[
chefox 17 points 3 years ago
What's the biggest lesson learned from D that you wish other languages would apply?

andralex 36 points 3 years ago
The scope statement. It's rare that I enjoy bragging about something, but I do like to brag that I invented a new control flow statement (which is awesome because they're so few!).
I think many languages implement some form of deferred execution, but most miss the point - Lisp's with-open-file, Java's try/finally, Go's defer, C#'s using are all sorely wanting.
]

Another one:

[ ... We have had an incredible time with compile-time evaluation in D for years, and I'm glad C++ "stole" that idea. ... ]

Another:

[
miotatsu 29 points 3 years ago
what has been your happiest moment with programming?

andralex 54 points 3 years ago

I remember moments when I'd run a little program again and again with slightly different inputs just to revel in the joy of having done the proper incantations that make the machine do this and that and the other, like a clumsy but loyal genie. I mentioned I wrote this floppy disk formatter - it gave me a lot of joy to be able to tell it the complicated sequence of things I wanted to get done, to see how it ended up carving magnetized trenches into the physical world.

]

Comment about the floppy driver formatter:
[
My first application was a keyboard driver in assembler. It used only 782 bytes of memory as opposed to the default of 4KB. I wrote it because I only had little RAM available.
My second application was a floppy disk formatter written in C++. It formatted 5.25'' floppies to 1.6 MB. Floppies were expensive and I couldn't afford many. Of course, they didn't retain information very well, but long-term retention wasn't at the top of my list :o).
]

About functional programming:
[
amzeratul 11 points 3 years ago
Lots of functional programming features are now available in languages such as C++ and D - what do you think are some of the functional features that we'll see incorporated in languages such as those over the next decade or two?

andralex[S] 17 points 3 years ago
There's talk about purity in C++, but beyond that I'm not sure whether there are plans for major FP-related additions.
Of the usual suspects present in FP languages, D notably misses pattern matching. It is in tension with OOP-style (first match vs. best match), and I'm not sure whether or not it's a fundamental feature of functional style. There are no plans to add such at this time.
]

[
p0nce 9 points 3 years ago
How much more productive do you think D can really be if adopted at C++ scale? Me and other enthusiasts I know are heavily biased by our positive solo experiences with it.

andralex[S] 10 points 3 years ago
Productivity and its variations are difficult to measure. Build speeds alone, at one order of magnitude speedup, are dramatic enough to exert a change of paradigm. For example, many people say dynamic languages are productive because they have the "right" execution model - save file, hit Refresh. If actual times for a compiled language drop to the point of offering the same model, I think a whole category of perceptions would change.
One thing I noticed with D is its "plasticity". Once you have a body of code that works in C++, the natural tendency is to be conservative about changing it: unit testing is tenuous, subtle failure scenarios upon changes are legion, not to mention build times etc. In D, it's a lot easier to mold and remold designs are you go because you know you wont be penalized for it.
]

[
Ingrater 10 points 3 years ago
Following the developement of D you get the feeling that way to many developement resources are spend on new features instead of finishing existing ones. For example a lot of work has been put into user defined attributes while other features are unfinished, unstable or not even usable (alias this, shared, export, structs). Whats your opinion on this?

andralex[S] 14 points 3 years ago
I agree that we should focus more on completing, streamlining, and using what we've got. This is happening already - it's been a while since quality has been at the top of our list, and the positive PR has been visibly improving.
Now that we have more resources there is some amount of parallel work we could do, and the mixed blessing with volunteer work is people work on what they find interesting, not necessarily what's best to do at the moment. This has been a challenge, but at the same time a good problem to have.
]

[
jfernand 10 points 3 years ago
Andralex,
No question to ask, but the comment that I carry the D language book with me at all times, and re-read it like it is a novel; I love every line of code I write in D, and the language and its design choices make me a better programmer the more I understand them. D simply makes sense, and it is a shame that it does not have more traction... I am an seasoned C/C++/Perl programmer, and D is in many ways the epitome of the history of languages, in my opinion.

andralex[S] 9 points 3 years ago
Thank you! No worry, we'll figure out the traction issue :o).
]

There's a lot more to the thread, check it out. Here it is again:

Andrei Alexandrescu's AMA on IAmA reddit



- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Are you a blogger with some traffic? Get Convertkit:

Email marketing for professional bloggers



Friday, April 14, 2017

Quick CMD one-liner like "which python" command - and more

By Vasudev Ram


Today, while browsing StackOverflow, I saw this question:

Is there an equivalent of 'which' on the Windows command line?

And I liked the second answer.

The first part of that answer had this solution:
c:\> for %i in (cmd.exe) do @echo.   %~$PATH:i
   C:\WINDOWS\system32\cmd.exe

c:\> for %i in (python.exe) do @echo.   %~$PATH:i
   C:\Python25\python.exe
Then I modified the one-liner above to add a few more commands to search for, and ran it:
$ for %i in (cmd.exe python.exe metapad.exe file_sizes.exe tp.exe alarm_clock.py
) do @echo.   %~$PATH:i
   C:\Windows\System32\cmd.exe
   D:\Anaconda2-4.2.0-32bit\python.exe
   C:\util\metapad.exe
   C:\util\file_sizes.exe
   C:\util\tp.exe
   C:\util\alarm_clock.py
Notice that I also included a .py file in the list of commands to search for, and it worked for that too. This must be because .py files are registered as executable files (i.e. executable by the Python interpreter) at the time of installing Python on Windows.

Of course, as a user comments in that SO post, this is not exactly the same as the which command, since you have to specify the .exe extension (for the commands you search for), and there are other extensions for executable files, such as .bat and others. But since the Python interpeter is normally named python.exe, this can be used as a quick-and-dirty way to find out which Python executable is going to be run when you type "python", if you have more than one of them installed on your system.

I had also written a simple Python tool roughly like the Unix which command, earlier:
A simple UNIX-like "which" command in Python

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Are you a blogger with some traffic? Get Convertkit:

Email marketing for professional bloggers



Wednesday, April 12, 2017

Interesting programming links of the week

By Vasudev Ram


I've been seeing a good number of interesting posts about programming topics lately, on various forums, particularly HN, so I thought of sharing them here (both for my readers and for myself to look up again later). Here are some selected posts or threads that I think are interesting to programmers:

Electron is flash for the desktop (2016) (josephg.com)

Web Development in Go (inburke.com)

Ask HN: Will Rust ever become a mainstream systems programming language?

Computer Programming Using GNU Smalltalk (2009) (canol.info)

Ask HN: What would you use to make cross-platform desktop application?

The reference D compiler is now open source (dlang.org)

The Power of Prolog (metalevel.at)

I also commented on some of the threads.

Enjoy.

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Are you a blogger with some traffic? Get Convertkit:

Email marketing for professional bloggers