Python Decorator Help - Accessing variables within the decorator from within the decorating function

Ignition
1

I'm playing with Python decorators and am a bit stumped. I wanted to see if I could create a decorator to help with logging and basic exception logging. I'm attempting to use this function to both create a contextualised LOGGER object to use in my functions foo and bar, but I can't get it working properly. The current version below creates a single logger "-shared.util.decorators-foo", instead of that and "-shared.util.decorators-bar".

import traceback
from java.lang import Throwable
LIBRARY = 'shared.util.decorators'
LOGGER = system.util.getLogger('bob')

def logger(LIBRARY):	
	"""
	Used to decorate functions to apply basic high level exception logging and provide a contextual LOGGER to use within the decorated function.
	
	Usage:
	from shared.util.decorators import logger
	@logger
	def foo(a,b):
		LOGGER.info('Running!')
		return a / b
	"""
	def decorator(f):
		def wrapper(*args, **kwargs):
			global LOGGER # removing this doesn't work, the root LOGGER object is used in this case when called from within the `f` function...
			LOGGER = system.util.getLogger('<COMPANY>-{}-{}'.format(LIBRARY, f.__name__))
			try:
				result = f(*args, **kwargs)
			except Throwable as e:
				LOGGER.error('Cause: {}, Traceback: {}'.format(e.cause, traceback.format_exc()))
			except Exception as e:
				LOGGER.error(traceback.format_exc())
				raise e
			return result
		return wrapper
	return decorator


@logger(LIBRARY)
def foo(a, b):
	LOGGER.info('{} / {}'.format(a,b))
	return float(a) / b

@logger(LIBRARY)
def bar(a, b):
	LOGGER.info('{} + {}'.format(a,b))
	return float(a) + b

I can use this, where I have to accept a LOGGER argument into every function I decorate, but I don't want to have to change the signature of every function...

import traceback
from java.lang import Throwable
LIBRARY = 'shared.util.decorators'
LOGGER = system.util.getLogger('bob')

def logger(LIBRARY):	
	"""
	Used to decorate functions to apply basic high level exception logging and provide a contextual LOGGER to use within the decorated function.
	
	Usage:
	from shared.util.decorators import logger
	@logger
	def foo(a,b):
		LOGGER.info('Running!')
		return a / b
	"""
	def decorator(f):
		def wrapper(*args, **kwargs):
			LOGGER = [None]
			LOGGER[0] = system.util.getLogger('SAGE-{}-{}'.format(LIBRARY, f.__name__))
			try:
				result = f(LOGGER[0], *args, **kwargs)
			except Throwable as e:
				LOGGER[0].error('Cause: {}, Traceback: {}'.format(e.cause, traceback.format_exc()))
			except Exception as e:
				LOGGER[0].error(traceback.format_exc())
				raise e
			return result
		return wrapper
	return decorator


@logger(LIBRARY)
def foo(LOGGER, a, b):
	LOGGER.info('{} / {}'.format(a,b))
	return float(a) / b

@logger(LIBRARY)
def bar(LOGGER, a, b):
	LOGGER.info('{} + {}'.format(a,b))
	return float(a) + b
2

"I" got it (stolen from SO). I commented the fixing lines with black magic references (5 lines in total). I'm still confused how it works...
If someone can explain these lines, I'd be very grateful!

As an aside, I'm sure someone will let me know how terrible of an idea this whole thing is :sweat_smile:

import traceback
from java.lang import Throwable
LIBRARY = 'shared.util.decorators'
CONTEXT = 'Scripting-{}'.format(LIBRARY)
LOGGER = system.util.getLogger('SAGE-{}'.format(CONTEXT))

def logger(context):	
	"""
	Used to decorate functions to apply basic high level exception logging and provide a contextual LOGGER to use within the decorated function.
	
	Usage:
	from shared.util.decorators import logger
	@logger('Scripting-shared.be.util')
	def foo(a,b):
		LOGGER.info('Running!')
		return a / b
	"""
	def decorator(f):
		LOGGER = system.util.getLogger('<COMPANY>-{}.{}'.format(context, f.__name__))
		# release some black magic to force the LOGGER to be available within f's function scope
		c = {'LOGGER': LOGGER}

		def wrapper(*args, **kwargs):
			# do some more black magic...
			f_globals = f.__globals__
			saved = {key: f_globals[key] for key in c if key in f_globals}
			f_globals.update(c)
			
			try:
				result = f(*args, **kwargs)
			except Throwable as e:
				LOGGER.error('Cause: {}, Traceback: {}'.format(e.cause, traceback.format_exc()))
			except Exception as e:
				LOGGER.error(traceback.format_exc())
				raise e
			finally:
				# do some more black magic...
				f_globals.update(saved)
			return result
		return wrapper
	return decorator

@logger(CONTEXT)
def logger_example(a, b):
	LOGGER.info('Running')
	return a / b

Apparently in Python 3 you just need to use the nonlocal keyword when declaring the variables and the black smoke dissappears. But alas, we are not there yet

3

Found this ona linked forum

Note that altering the globals is not thread safe, and any transient calls to other functions in the same module will also still see this same global.

**kwargs seem to be a safer way

c = 'Message'

def decorator_factory(value):
    def msg_decorator(f):
    def inner_dec(*args, **kwargs):
        kwargs["var"] = value
        res = f(*args, **kwargs)
        return res
    return inner_dec
return msg_decorator

@decorator_factory(c)
def msg_printer(*args, **kwargs):
    print kwargs["var"]

msg_printer()
4

The thing I don't want to do is to have to modify every function signature though :confused:

5

I mean, aslong you dont create another logger in the same way somewhere else. It shouldnt be a problem.

You might want to give it a more unique name than logger though, just in case

6

I honestly don’t think you need the black magic. I haven’t proved it yet, but I’m working on it.

7

What exactly are you trying to achieve ?

8

An alternative way to do decorators with a class and functools here, I find it a little bit easier to work with but in reality anytimes I touch decorators I need a small referesher lol- Script Benchmarking with Decorators Example - #4 by dkhayes117

If you note the first answer in the link @dkhayes117 returns the function result if there is one or None if there is an error and you can use your control flow like that.

I modified that a bit, I always return a Response object which is just

class Response:
    success: bool - true if no errors, false otherwise
    result: Any, whatever the result of the function was, None if success was False
    errorType:  str - python  or java to indicate what type of error
    errorMsg: the relevant error message (depends on if python or java)

So I decorate my most basic Business Logic with this, and then I can do control flow from that - was it successful - do X, was it a python error - I probably messed up code and need to tell them to alert me, if it's java, it may just be a expected database error and I can show a useful error to the end user etc. Seems like this may sort of be what you are trying to do, not sure your end goal though. Hope this is helpful.

1 Like
9

I also use some magic to get a representational string of the function with this

# Format the failed call into the same format that it would written in code
try: fnPath = '%s.%s' % (fn.func_code.co_filename.split(':')[1].replace('>',''), fn.__name__)
except: fnPath = fn.__name__

argsFmt = ','.join(['%r' % x for x in values['args']])
kwargsFmt = ','.join(['%s=%r' % (k,v) for k,v in values['kwargs'].items()])

callString = '%s(%s)' % (fnPath, ','.join(filter(None,(argsFmt,kwargsFmt))))
10

You don't need the brackets in the join calls, it can take a genexpr. Might as well use it.

1 Like