Paul's Programming Notes     Archive     Feed     Github

PHP - max_execution_time doesn't work for socket operations

In new relic transaction logs, I was seeing some PHP requests last as long as an hour. It turns out that time spent waiting on sockets doesn't apply toward max_execution_time, and loops that involved waiting on sockets could end up taking a really long time without timing out.

PHP docs have this to say about max_execution_time:
"The set_time_limit() function and the configuration directive max_execution_time only affect the execution time of the script itself. Any time spent on activity that happens outside the execution of the script such as system calls using system(), stream operations, database queries, etc. is not included when determining the maximum time that the script has been running. This is not true on Windows where the measured time is real."

The solution ended up being setting request_terminate_timeout in php-fpm.

Python - Cachetools LRUCache KeyError

If your application is threaded and you're getting a "KeyError" while using the non-decorator version of cachetool's LRUCache, then you need to put whatever is manipulating the cache object inside of a lock. Also, since LRUCache is modified when values are gotten from it, you will also need to make sure you're locking when you get values from cache too. If you can use the decorator version of LRUCache, that's preferred since it has built-in locking.

Here's an example of the error:

[2016-08-10 22:36:30 +0000] [1] [ERROR] [blah/blah_user] "my_key"
Traceback (most recent call last):
File "/usr/local/lib/python3.4/site-packages/thriftpy/server.py", line 92, in handle
self.processor.process(iprot, oprot)
File "/usr/local/lib/python3.4/site-packages/thriftpy/thrift.py", line 244, in process
self.handle_exception(e, result)
File "/usr/local/lib/python3.4/site-packages/thriftpy/thrift.py", line 241, in process
result.success = call()
File "/usr/local/lib/python3.4/site-packages/thriftpy/thrift.py", line 205, in call
*(args.__dict__[k] for k in api_args)
File "/opt/service/blah/dispatcher.py", line 932, in get
return blah.to_thrift()
File "/opt/service/blah/model.py", line 383, in to_thrift
profile = self.profile
File "/opt/service/blah/model.py", line 360, in profile
return service_callers.get_profile(TProfileIdentifiers(id=str(self.profile_id)))
File "/opt/service/blah/util.py", line 121, in func_wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.4/site-packages/dogpile/cache/region.py", line 1053, in decorate
should_cache_fn)
File "/usr/local/lib/python3.4/site-packages/dogpile/cache/region.py", line 657, in get_or_create
async_creator) as value:
File "/usr/local/lib/python3.4/site-packages/dogpile/core/dogpile.py", line 158, in __enter__
return self._enter()
File "/usr/local/lib/python3.4/site-packages/dogpile/core/dogpile.py", line 98, in _enter
generated = self._enter_create(createdtime)
File "/usr/local/lib/python3.4/site-packages/dogpile/core/dogpile.py", line 149, in _enter_create
created = self.creator()
File "/usr/local/lib/python3.4/site-packages/dogpile/cache/region.py", line 630, in gen_value
self.backend.set(key, value)
File "/usr/local/lib/python3.4/site-packages/dogpile/cache/backends/memory.py", line 76, in set
self._cache[key] = value
File "/usr/local/lib/python3.4/site-packages/cachetools/lru.py", line 19, in __setitem__
cache_setitem(self, key, value)
File "/usr/local/lib/python3.4/site-packages/cachetools/cache.py", line 51, in __setitem__
self.popitem()
File "/usr/local/lib/python3.4/site-packages/cachetools/lru.py", line 33, in popitem
return (key, self.pop(key))
File "/usr/local/lib/python3.4/site-packages/cachetools/abc.py", line 36, in pop
raise KeyError(key)
KeyError: "my_key"
view raw gistfile1.txt hosted with ❤ by GitHub
And an example of the fix: https://bitbucket.org/zzzeek/dogpile.cache/pull-requests/32/add-a-cachetools-lru-lfu-in-memory-backend/diff#comment-22242704

Javascript's Round vs PHP's Round

Today I learned that by default PHP rounds differently than javascript.

PHP (using the default PHP_ROUND_HALF_UP)

php > echo round(-1.5);
-2
"Round val up to precision decimal places away from zero, when it is half way there. Making 1.5 into 2 and -1.5 into -2."
https://secure.php.net/manual/en/function.round.php

Javascript

Math.round(-1.5);
-1
"For negative numbers, if the decimal portion is exactly -0.5, the return value is the smallest integer that is greater than the number."
https://msdn.microsoft.com/en-us/library/5cza0web(v=vs.94).aspx

Javascript Data Tables