36 Commits

Author SHA1 Message Date
long2ice
de1bde39fd update version 2021-07-23 09:38:47 +08:00
long2ice
8490ad36f0 Merge pull request #25 from dveleztx/master
This fixes #24.
2021-07-23 09:38:04 +08:00
David Velez
57fe4ce24b Updated changelog to 1.4 from 1.5 2021-07-22 10:54:10 -05:00
David Velez
3dc2b53e41 Updated changelog and version for the project. 2021-07-22 10:47:02 -05:00
David Velez
2dd37b09ab This fixes #24. Looking at aioredis library, the client.py keyword for set is now 'ex', not 'expire'. Tested this fix and fast-cache now works without issue. 2021-07-22 10:17:52 -05:00
long2ice
0bc8c6c20e Merge pull request #22 from heliumbrain/master
Adapt to aioredis 2.0
2021-07-16 10:02:10 +08:00
heliumbrain
9e3c9816c5 Update README.md
Updated readme to reflect the changes in aioredis 2.0
2021-07-15 23:15:01 +02:00
long2ice
7c7aa26a88 update deps 2021-07-15 10:00:27 +08:00
long2ice
1edb0ba1fe Create FUNDING.yml 2021-04-30 16:49:36 +08:00
long2ice
7c2007847f Merge pull request #8 from rushilsrivastava/patch-2
Use FastAPI's built in jsonable_encoder
2021-04-30 16:48:07 +08:00
long2ice
80de421a2a Merge pull request #18 from Agri1988/master
add timezone serializer to coder.JsonEncoder for datetime.datetime instances
2021-04-29 09:36:48 +08:00
alexandr.yagolovich
eb55b01be9 timezone serializer to coder.JsonEncoder for datetime.datetime instances 2021-04-28 20:44:35 +03:00
long2ice
8573eeace6 fix ci 2021-03-20 14:47:46 +08:00
long2ice
1d0c245a70 - Fix default expire for memcached. (#13)
- Update default key builder. (#12)
2021-03-20 14:42:29 +08:00
long2ice
c665189d90 Merge pull request #7 from shershen08/patch-1
Update README.md
2021-01-12 09:50:25 +08:00
Mikhail Kuznetcov
ba7276ba98 Update README.md
add other params description
2021-01-11 17:37:39 +01:00
Rushil Srivastava
30e5246cf5 Use FastAPI's built in jsonable_encoder 2021-01-11 03:11:35 -08:00
Mikhail Kuznetcov
cdae610432 Update README.md
add explanation about expire
2021-01-10 16:26:41 +01:00
long2ice
9157412d6a Merge pull request #5 from rushilsrivastava/patch-1
Add minimum dependencies version
2021-01-10 19:59:15 +08:00
Rushil Srivastava
a9b0b9d913 add min dependencies 2021-01-10 03:53:26 -08:00
long2ice
fec3c78291 bug fix 2021-01-06 20:00:58 +08:00
long2ice
361a25857b bug fix 2021-01-06 19:57:16 +08:00
long2ice
6f5d4900a9 bug fix 2021-01-06 14:41:46 +08:00
long2ice
7f7252d151 bug fix 2021-01-06 10:44:22 +08:00
long2ice
c9e03ed9af update pypi.yml 2021-01-06 10:38:30 +08:00
long2ice
7f2ebfc494 Merge remote-tracking branch 'origin/master'
# Conflicts:
#	fastapi_cache/decorator.py
#	poetry.lock
2021-01-06 10:35:06 +08:00
long2ice
a42fdaf632 Fix cache key builder. 2021-01-06 10:34:30 +08:00
long2ice
9ca5b0fa9a update version 2020-12-28 20:04:51 +08:00
long2ice
75b4547963 add no-store 2020-12-28 20:04:09 +08:00
long2ice
3134e5f67c update README.md 2020-11-12 13:04:50 +08:00
long2ice
0fd9e53e06 update memory cache 2020-11-10 10:34:52 +08:00
long2ice
ff8b4b385d Merge remote-tracking branch 'origin/master' 2020-11-09 20:32:30 +08:00
long2ice
fd3ffa0d21 update memcached 2020-11-09 20:32:23 +08:00
long2ice
80536a1429 fix ci error 2020-11-04 12:58:01 +08:00
long2ice
e483e0dc55 add clear method 2020-11-03 18:08:06 +08:00
long2ice
dc2ac9cc90 Add default config when init
update JsonEncoder
2020-10-16 16:55:33 +08:00
17 changed files with 632 additions and 457 deletions

2
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,2 @@
custom: ["https://sponsor.long2ice.cn"]

View File

@@ -8,6 +8,9 @@ jobs:
- uses: actions/setup-python@v2
with:
python-version: "3.x"
- uses: dschep/install-poetry-action@v1.3
- name: Install and configure Poetry
uses: snok/install-poetry@v1.1.1
with:
virtualenvs-create: false
- name: CI
run: make ci

View File

@@ -11,7 +11,10 @@ jobs:
- uses: actions/setup-python@v1
with:
python-version: '3.x'
- uses: dschep/install-poetry-action@v1.3
- name: Install and configure Poetry
uses: snok/install-poetry@v1.1.1
with:
virtualenvs-create: false
- name: Build dists
run: make build
- name: Pypi Publish

View File

@@ -2,6 +2,25 @@
## 0.1
### 0.1.5
- Fix setting expire for redis (#24)
- Update expire key
### 0.1.4
- Fix default expire for memcached. (#13)
- Update default key builder. (#12)
### 0.1.3
- Fix cache key builder.
### 0.1.2
- Add default config when init.
- Update JsonEncoder.
### 0.1.1
- Add in-memory support.

View File

@@ -32,7 +32,10 @@ check: deps
test: deps
$(py_warn) pytest
build: deps
build: clean deps
@poetry build
clean:
@rm -rf ./dist
ci: check test

View File

@@ -69,15 +69,27 @@ async def index(request: Request, response: Response):
@app.on_event("startup")
async def startup():
redis = await aioredis.create_redis_pool("redis://localhost", encoding="utf8")
redis = aioredis.from_url("redis://localhost", encoding="utf8", decode_responses=True)
FastAPICache.init(RedisBackend(redis), prefix="fastapi-cache")
```
### Initialization
Firstly you must call `FastAPICache.init` on startup event of `fastapi`, there are some global config you can pass in.
### Use `cache` decorator
If you want cache `fastapi` response transparently, you can use `cache` as decorator between router decorator and view function and must pass `request` as param of view function.
Parameter | type, description
------------ | -------------
expire | int, states a caching time in seconds
namespace | str, namespace to use to store certain cache items
coder | which coder to use, e.g. JsonCoder
key_builder | which key builder to use, default to builtin
And if you want use `ETag` and `Cache-Control` features, you must pass `response` param also.
You can also use `cache` as decorator like other cache tools to cache common function result.
@@ -95,6 +107,8 @@ async def index(request: Request, response: Response):
### Custom key builder
By default use builtin key builder, if you need, you can override this and pass in `cache` or `FastAPICache.init` to take effect globally.
```python
def my_key_builder(
func,
@@ -116,7 +130,7 @@ async def index(request: Request, response: Response):
### InMemoryBackend
`InMemoryBackend` only support in single node instead of distributed environment.
`InMemoryBackend` store cache data in memory and use lazy delete, which mean if you don't access it after cached, it will not delete automatically.
## License

View File

@@ -12,7 +12,7 @@ app = FastAPI()
ret = 0
@cache(expire=1)
@cache(namespace="test", expire=1)
async def get_ret():
global ret
ret = ret + 1
@@ -20,11 +20,16 @@ async def get_ret():
@app.get("/")
@cache(expire=2)
@cache(namespace="test", expire=2)
async def index(request: Request, response: Response):
return dict(ret=await get_ret())
@app.get("/clear")
async def clear():
return await FastAPICache.clear(namespace="test")
@app.on_event("startup")
async def startup():
FastAPICache.init(InMemoryBackend(), prefix="fastapi-cache")

View File

@@ -1,17 +1,34 @@
from typing import Callable
from fastapi_cache.coder import Coder, JsonCoder
from fastapi_cache.key_builder import default_key_builder
class FastAPICache:
_backend = None
_prefix = None
_expire = None
_init = False
_coder = None
_key_builder = None
@classmethod
def init(cls, backend, prefix: str = "", expire: int = None):
def init(
cls,
backend,
prefix: str = "",
expire: int = None,
coder: Coder = JsonCoder,
key_builder: Callable = default_key_builder,
):
if cls._init:
return
cls._init = True
cls._backend = backend
cls._prefix = prefix
cls._expire = expire
cls._coder = coder
cls._key_builder = key_builder
@classmethod
def get_backend(cls):
@@ -25,3 +42,16 @@ class FastAPICache:
@classmethod
def get_expire(cls):
return cls._expire
@classmethod
def get_coder(cls):
return cls._coder
@classmethod
def get_key_builder(cls):
return cls._key_builder
@classmethod
async def clear(cls, namespace: str = None, key: str = None):
namespace = cls._prefix + ":" + namespace if namespace else None
return await cls._backend.clear(namespace, key)

View File

@@ -14,3 +14,7 @@ class Backend:
@abc.abstractmethod
async def set(self, key: str, value: str, expire: int = None):
raise NotImplementedError
@abc.abstractmethod
async def clear(self, namespace: str = None, key: str = None) -> int:
raise NotImplementedError

View File

@@ -1,6 +1,6 @@
import time
from asyncio import Lock
from dataclasses import dataclass
from threading import Lock
from typing import Dict, Optional, Tuple
from fastapi_cache.backends import Backend
@@ -29,18 +29,31 @@ class InMemoryBackend(Backend):
return v
async def get_with_ttl(self, key: str) -> Tuple[int, Optional[str]]:
with self._lock:
async with self._lock:
v = self._get(key)
if v:
return v.ttl_ts - self._now, v.data
return 0, None
async def get(self, key: str) -> str:
with self._lock:
async with self._lock:
v = self._get(key)
if v:
return v.data
async def set(self, key: str, value: str, expire: int = None):
with self._lock:
async with self._lock:
self._store[key] = Value(value, self._now + expire)
async def clear(self, namespace: str = None, key: str = None) -> int:
count = 0
if namespace:
keys = list(self._store.keys())
for key in keys:
if key.startswith(namespace):
del self._store[key]
count += 1
elif key:
del self._store[key]
count += 1
return count

View File

@@ -5,15 +5,18 @@ from aiomcache import Client
from fastapi_cache.backends import Backend
class MemcacheBackend(Backend):
class MemcachedBackend(Backend):
def __init__(self, mcache: Client):
self.mcache = mcache
async def get_with_ttl(self, key: str) -> Tuple[int, str]:
return 0, await self.mcache.get(key.encode())
return 3600, await self.mcache.get(key.encode())
async def get(self, key: str):
return await self.mcache.get(key, key.encode())
async def set(self, key: str, value: str, expire: int = None):
return await self.mcache.set(key.encode(), value.encode(), exptime=expire)
return await self.mcache.set(key.encode(), value.encode(), exptime=expire or 0)
async def clear(self, namespace: str = None, key: str = None):
raise NotImplementedError

View File

@@ -19,4 +19,11 @@ class RedisBackend(Backend):
return await self.redis.get(key)
async def set(self, key: str, value: str, expire: int = None):
return await self.redis.set(key, value, expire=expire)
return await self.redis.set(key, value, ex=expire)
async def clear(self, namespace: str = None, key: str = None) -> int:
if namespace:
lua = f"for i, name in ipairs(redis.call('KEYS', '{namespace}:*')) do redis.call('DEL', name); end"
return await self.redis.eval(lua)
elif key:
return await self.redis.delete(key)

View File

@@ -1,6 +1,43 @@
import datetime
import json
import pickle # nosec:B403
from decimal import Decimal
from typing import Any
from fastapi.encoders import jsonable_encoder
import dateutil.parser
CONVERTERS = {
"date": dateutil.parser.parse,
"datetime": dateutil.parser.parse,
"decimal": Decimal,
}
class JsonEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, datetime.datetime):
if obj.tzinfo:
return {"val": obj.strftime("%Y-%m-%d %H:%M:%S%z"), "_spec_type": "datetime"}
else:
return {"val": obj.strftime("%Y-%m-%d %H:%M:%S"), "_spec_type": "datetime"}
elif isinstance(obj, datetime.date):
return {"val": obj.strftime("%Y-%m-%d"), "_spec_type": "date"}
elif isinstance(obj, Decimal):
return {"val": str(obj), "_spec_type": "decimal"}
else:
return jsonable_encoder(obj)
def object_hook(obj):
_spec_type = obj.get("_spec_type")
if not _spec_type:
return obj
if _spec_type in CONVERTERS:
return CONVERTERS[_spec_type](obj["val"])
else:
raise TypeError("Unknown {}".format(_spec_type))
class Coder:
@@ -16,11 +53,11 @@ class Coder:
class JsonCoder(Coder):
@classmethod
def encode(cls, value: Any):
return json.dumps(value)
return json.dumps(value, cls=JsonEncoder)
@classmethod
def decode(cls, value: Any):
return json.loads(value)
return json.loads(value, object_hook=object_hook)
class PickleCoder(Coder):

View File

@@ -1,30 +1,14 @@
from functools import wraps
from typing import Callable, Optional, Type
from starlette.requests import Request
from starlette.responses import Response
from fastapi_cache import FastAPICache
from fastapi_cache.coder import Coder, JsonCoder
def default_key_builder(
func,
namespace: Optional[str] = "",
request: Request = None,
response: Response = None,
*args,
**kwargs,
):
prefix = FastAPICache.get_prefix()
cache_key = f"{prefix}:{namespace}:{func.__module__}:{func.__name__}:{args}:{kwargs}"
return cache_key
from fastapi_cache.coder import Coder
def cache(
expire: int = None,
coder: Type[Coder] = JsonCoder,
key_builder: Callable = default_key_builder,
coder: Type[Coder] = None,
key_builder: Callable = None,
namespace: Optional[str] = "",
):
"""
@@ -39,9 +23,23 @@ def cache(
def wrapper(func):
@wraps(func)
async def inner(*args, **kwargs):
request = kwargs.get("request")
nonlocal coder
nonlocal expire
nonlocal key_builder
copy_kwargs = kwargs.copy()
request = copy_kwargs.pop("request", None)
response = copy_kwargs.pop("response", None)
if request and request.headers.get("Cache-Control") == "no-store":
return await func(*args, **kwargs)
coder = coder or FastAPICache.get_coder()
expire = expire or FastAPICache.get_expire()
key_builder = key_builder or FastAPICache.get_key_builder()
backend = FastAPICache.get_backend()
cache_key = key_builder(func, namespace, *args, **kwargs)
cache_key = key_builder(
func, namespace, request=request, response=response, args=args, kwargs=copy_kwargs
)
ttl, ret = await backend.get_with_ttl(cache_key)
if not request:
if ret is not None:
@@ -54,7 +52,6 @@ def cache(
return await func(request, *args, **kwargs)
if_none_match = request.headers.get("if-none-match")
if ret is not None:
response = kwargs.get("response")
if response:
response.headers["Cache-Control"] = f"max-age={ttl}"
etag = f"W/{hash(ret)}"

View File

@@ -0,0 +1,25 @@
import hashlib
from typing import Optional
from starlette.requests import Request
from starlette.responses import Response
def default_key_builder(
func,
namespace: Optional[str] = "",
request: Optional[Request] = None,
response: Optional[Response] = None,
args: Optional[tuple] = None,
kwargs: Optional[dict] = None,
):
from fastapi_cache import FastAPICache
prefix = f"{FastAPICache.get_prefix()}:{namespace}:"
cache_key = (
prefix
+ hashlib.md5( # nosec:B303
f"{func.__module__}:{func.__name__}:{args}:{kwargs}"
).hexdigest()
)
return cache_key

835
poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
[tool.poetry]
name = "fastapi-cache2"
version = "0.1.1"
version = "0.1.5"
description = "Cache for FastAPI"
authors = ["long2ice <long2ice@gmail.com>"]
license = "Apache-2.0"
@@ -18,8 +18,9 @@ include = ["LICENSE", "README.md"]
python = "^3.7"
fastapi = "*"
uvicorn = "*"
aioredis = {version = "*", optional = true}
aioredis = {version = ">=2.0.0b1", optional = true}
aiomcache = {version = "*", optional = true}
python-dateutil = "*"
[tool.poetry.dev-dependencies]
flake8 = "*"