U cĹ@sddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl m Z m Z ddl mZddlmZddlmZmZddlmZddlZddlZddlZddlZddlmZddlmZdd lmZmZdd lm Z dd l!m"Z"dd l#m$Z$dd l%Tddl&m'Z(ddl)Z#Gddde*Z*ddZ+Gddde,Z-ddZ.Gddde Z/e*e0de1j2dZ3e/e3_4de3_5de3_6ge3_7ge3_8ge3_9ge3_:ge3_;ge3_m?Z?m@Z@ddlAmBZBddlAmCZCddlDZDdd d!ZEGd"d#d#ZFeFZGe eHZId$d%ZJd&d'ZKdd(d)ZLeMe1d*rpe1jNd+d,d-krpe1jNd+OZPePQd,eDjRfePeG_DeSjTUd.reGjDdkreVd/e3d0e1jWd1d2ZXeMe1d*r6e1jNd+d,d-kr6e1jNd+d3ZYe1jNd+Ud4rd5e1jNd+d4d6eYZZeBeZd7d1d8Z[e[eXd9<n"d:e1jNd+d;dl]m^Z^d?e^__e?feXZAeCe1jWd@Z`eCe1jWd@ZaeSjbce1j2dArede1j2dAZeeefge1_hW5QRXeMe1d*re1jNd+d,d-kre-eGjDdBZin(e1jNd+d,dCkre-ddDZineVdEne-ZieGe3_GdFdGZjejZkekjldddHdIdJZme3neVdKdLZoe3peVeoee1j2dMZqee1j2dNZrddOlsmtZteqcretueqve3_wercretuerve3_xdPdQdRdSdTdUdVdWdXdYdZd[d\d]d^d_ge3_yd`daZ5dbdcZzdddeZ{dfdgZ|dhdiZ6djdkZ}dldmZ~dndoZee5ee_5ee}ee_}ee~ee_~eeee_ee|ee_|ee|ee_ee6ee_6dpdqZeeee_drdsZeeee_ddvdwZdxdyZddzd{Ze3jd|d}Zd~dZddZddZddZdddZddZddZddZddZe3jddZddZe3jddZe3jdddZdddZdddgdddZddZdddZddZddZddZddZGdddZddZdddZddZddZddZddZddZedddZeedddZdS)N) OrderedDict defaultdict)Path)quote)urlparseparse_qs)Celery)SHA256) PKCS1_v1_5)abortResponse) JSONEncoder) HTTPExceptionfields)*)basecs$eZdZfddZddZZS)Flaskcstj||dSN)super__init__selfargskwargs __class__./app.pyr%szFlask.__init__c sfdd}|S)Ncsndd}|j}|dd}|dkr6d|}n}|dkrP|d|j}j|||ft||S)Nendpoint.index/)pop __module__split__name__ add_url_ruleprint)fr module_pathZ module_nameurloptionsrulerrr decorator)s zFlask.route..decoratorr)rr/r.r0rr-rroute(sz Flask.route)r'r% __qualname__rr1 __classcell__rrrrr$s rcOs|rt||dSr)r))Zcondrrrrr print_debug:sr4c@s^eZdZdZdZdZdZdZdZdJddddZ ddZ dd Z d d Z d d Z dKddZddZddZddZddZeeedddZeedddZedLddZd d!Zd"d#Zd$d%Zd&d'Zd(d)Zd*d+Zd,d-ZdMd.d/Zd0d1Z d2d3Z!d4d5Z"d6d7Z#d8d9Z$d:d;Z%dd?Z'd@dAZ(dBdCZ)dDdEZ*dNdFdGZ+dHdIZ,dS)OCmfCacheNFreturncCsBt|_t|_||_||_|jr>|jdd|jdddS)Nstat_hitr stat_miss)dict cache_objcache_invalidateredis_dbmemoryincrby)rr=r>rrrrGszCmfCache.__init__cCs>ttjp |jdtjd|jr*|jnt|_t|_ dS)NzCmfCache.flushall!!!file) r4configDEBUGsysstderrr=flushdbr:r;r<rrrrflushallQs  zCmfCache.flushallcCsttjp |jdtjdtt}|j}|jj dddD]p}t|jd|dtjd|dd  }|j |}|d kr~q:t |}|d }|d |||<||q:|||d S) u)Инвалидируем весь jscachezCmfCache.flush_jscache!!!r@z obj:jshash:*countz"CmfCache.flush_jscache invalidate z!!!Ncurrent_person_idjsver)r4rBrCrDrErr:r=pipeline scan_iterdecodegetpickleloadsdeleteexecuteemit)remit_msgpipekeyjshashjsresjspersonrrr flush_jscacheYs    zCmfCache.flush_jscachecCsttjp |jdtjdd}|j}|jjdddD]"}|drFq6|d7}| |q6| ttjpn|jd |d tjdd S) u+Инвалидируем весь obj cachezCmfCache.flush_all_obj!!!r@rzobj:*rIrJs obj:jshashzCmfCache.flush_all_obj: z recordsN) r4rBrCrDrEr=rOrP startswithrUrV)rrKrYrZrrr flush_all_objks   zCmfCache.flush_all_objcsddl|}fdd|D]}|d|}qd|krP|dt|d}t|D]*\}}|dkrnq\|d|d|}q\|jrd|dd d |}|d d}n|d|}t |jd ||S) Nrc shddl}ddlm}t|tjjr4|jr0|j}nd}d}t|t t t frt|t r|dddkrz.|j |}||j}|jddd}|}Wn"tk r}zW5d}~XYnXt |}nt|tr|D]}||d}qnt|tr.|D]2\}}||}|r"|d |}q|d}qn6|dkr>d }n&d t|jd t|}|S) Nr)timezonez...Z20)secondZ microsecond,:Nonez todo2:obj:)Zdateutil.parserdatetimerb isinstancecmfrCmfType is_definedvaluestrintfloatparserparseZ astimezoneutcreplace Exceptionlistr:itemstyper'md5rSdumps hexdigest)valdateutilrbresZval1evk_all2strhashlibrrr|s<        &zCmfCache.hash.._all2str|idz|id:rgzHASH:iz|md5: zDEBUG_CACHE HASH:) rrosortedrxrCrzencoder|rur4)rprefixrrsr}rZrrrhashxs"+&z CmfCache.hashc Cs|dkrt}|s|S||jtt|j}|jD]}t||jsJq8t |t jj rzt||jd}|r|j ||dq8t |t jj t jjfr8t||jd}|r|D]}|j ||dq|D]"}d|jd|j}||qq8|S)N)rref__)setaddrgetattrmodels class_namervalueshasattr issubclassrkCmfRelationBasesimple_objects_id_recursive CmfM2MBaseCmfBackrefBaserelated_models) robjrZmodelZ field_clsZ field_valuerel_obj rel_modelrefrrrrs,    z$CmfCache.simple_objects_id_recursivecCs|sgS|jjg}|jddD]}t|tjjrB|||jt|tjj tjj fr|jrz|jD]}|||qd| D]"}d|jd|j }| |qqtt|}|S)u Собираем ид связанных обьектов 2 уровня для настройки инвалидации :param obj: :return: Trmrr)rrnrrjrkrrextendobjects_id_recursiverrrrappendrwr)rrrfieldrrrrrrrs    zCmfCache.objects_id_recursivecCst|}t|tjkrd}t|tjjs4t |r@d|}nJ|rt|t rt|dtjjsnt |drd|dj dt|d}t dt|d|dd d S|jr|jd ||dS|jr||j|<dSdS) Nrc, r[]uRDEV: Слишком большой объект для кэширования — u байтT) debug_onlyFobj:)rSr{lenrBZCACHE_MAX_OBJ_SIZErjrkr BaseModel dataclassesZ is_dataclassrwr cmf_alertr=rr>r;)rrZr}Zstr_valZdetailrrr _obj_dict_sets&    zCmfCache._obj_dict_setcGsrt}|D]}|d|q |jrD|jj|t|kr>dSdSn*|jrj|D]}|j|sNdSqNdSdSdS)NrTF)rrr=existsrr>r;rR)rkeysZhkeysrZrrr_obj_dict_existss zCmfCache._obj_dict_existscCsX|jr>|jd|}|dk r.|jdn |jd|S|jrP|j|SdSdS)Nrr8r9)r=rRr?r>r;rrZrrrr _obj_dict_gets  zCmfCache._obj_dict_getr[ inv_payloadcCs|d|S)N::rrrrr%_jshash_invalidate_confirm_member_keysz.CmfCache._jshash_invalidate_confirm_member_keyZ member_keycCs|jdddS)Nrr_)maxsplitr&rrrr+_jshash_invalidate_confirm_member_key_split#sz4CmfCache._jshash_invalidate_confirm_member_key_splitcCs |r|n |dd}d|S)Nrgzjscache-invalidate-confirm-set:r)cache_idr[set_keyrrr"_jshash_invalidate_confirm_set_key'sz+CmfCache._jshash_invalidate_confirm_set_keyc Cstt}|D]Z\}}td||d|gd|D]0\}}|dr8||j|d|||q8q|jr|D]\}}|jj |f|qzdS)Nz invalidate-)room event_personsjshash:r[) rrwrxcmf_emit_eventr`rrrr=sadd) rZmsgsZ confirmations person_idinv_dictr[rr confirm_listrrrrW.s  z CmfCache.emitcCsb|jr^|r^|dd}|j|d}t}|D]}|||d|dq,|jj|f|dS)Nrrr_)r=rrwrrsrem)rZ confirm_dictZ sample_jshashZ key_cache_idrZconfirmrrrjshash_invalidate_confirm=s   z"CmfCache.jshash_invalidate_confirmcs.jr*|}fddj|DSdS)Ncsg|]}|qSr)rrQ).0memberrGrr Nsz;CmfCache.jshash_invalidate_confirm_list..)r=rsmembers)rrrrrGrjshash_invalidate_confirm_listIs    z'CmfCache.jshash_invalidate_confirm_listc Cs|sdS|jr|jjdd|Di}|D]}|jrPd|krPtd|t|jd||jd|}|r.t|jd||jrtd||D]}|d }|jrd|krtd |d |t|jd ||jd |} | dkrqt | } | r| d} | |kr.t || <|dkrZd t dd|j ddD} nd} | dd|d|d| } | || |<|jd |q|jd|||q.n.|jr|D]}||jkr|j|=qndSdS)NcSsg|]}d|qS)rrrrZrrrrVsz+CmfCache._obj_dict_mdel..rKz#DEBUG_CACHE JSHASH: try invalidate z*DEBUG_CACHE JSHASH: try invalidate:jshash:zinvalidate:jshash:z*DEBUG_CACHE JSHASH: got invalidate:jshash:z#DEBUG_CACHE JSHASH: got invalidate utf-8z"DEBUG_CACHE JSHASH: do invalidate  z)DEBUG_CACHE JSHASH: do invalidate jshash:rrMupdatercSsg|]}|dkr|qS))Z cache_fieldscmf_modified_at cmf_viewed_atZcmf_modified_by_idr)rrrrrrusT) is_changedZONLYFORUPDATESrN)r=rUrCrr4rrQrRrSrTr:joinrrrWr>r;) rrrZobj_idactionrXrZZinvjsr[r\r]Zchanged_fieldsrrrr_obj_dict_mdelRsT         zCmfCache._obj_dict_mdelc Cs|jrN|j6}|D]\}}|jd|f|q|W5QRXnD|jr|D]2}||jkr||j|||qX|||j|<qXndSdSN invalidate:)r=rOrxrrVr>r<r)rrrYrZrrrr_invalidate_dict_msets  zCmfCache._invalidate_dict_msetcCsH|jr.|jd|}|r*dd|DS|S|jr@|j|SdSdS)NrcSsg|]}|dqS)r)rQrrrrrsz1CmfCache._invalidate_dict_get..)r=rr>r<rR)rnamerrrr_invalidate_dict_gets zCmfCache._invalidate_dict_getc Cs|jrT|j<}|D]$\}}|r|jjd|f|q|W5QRXnJ|jr|D]4\}}|D]&}||j|gkrn|j||qnqbndSdSr) r=rOrxrrVr>r<rRremove)rrrYrrZkey_listrZrrr_invalidate_dict_mdels zCmfCache._invalidate_dict_mdelc srtjdrdt_dStr>dt_tjdt dStj  krtdt_tjdtj d dSifddfdd }|rt |t st|d r||d }nx|rt |t rt|d d rd }||nN|dk rd}t|tkr2t|tkr2ttkr2n|s*d}d}ntdd}|r|drtjd||d kr|dD]:}||jd|jdfkrfd}|jqqf|sg} |dr| |d7} |dr| |d7} |dr| |d7} | sdg} | D]} | dkrd} |dD]h}|dd } || krTd|d| dn*d | d!|d | d!| dqq|d"D]} d| dq|r|dsƈd|djrD]} tjd#| d$q҈|sdt_d%tkrdtjrdtjd&dtjjrLtd&tjd'tjfidS)(NNO_CACHETz%DEBUG_CACHE ADD SKIP CACHE LOCKED_BY z#DEBUG_CACHE ADD SKIP too old trans rcs$|krt|<|dSr)rwr)bind)rrZrrbind_adds zCmfCache.add..bind_addcsdt|tr|}n|g}g}|D]@}t|jtjjr>|}nt|}|D] }|qPqdS)u Собираем списки ключей инвалидации :param val: :param invalidate_dict: :return: N)rjrwrrkrrlrr)rZobj_listritemr)rrrrbind_add_recursives  z(CmfCache.add..bind_add_recursiverrrrwZcustomemptyZEmptyuGНевозможный объект для хранения в кешеFrzDEBUG_CACHE WHERE: z.idz.codeparentslistsZcache_clustersZScopeAll)ZNullZNULLZnullrhZnoneNrhr where_ScopeAll__insdelwhere_rrzDEBUG_CACHE BIND: z = r[#DEBUG_CACHE JSHASH: add invalidate r)osenvironrRgno_jscache_force CMF_CACHEcache_is_lockedr4rCcache_locked_bycache_transaction_startget_last_invalidaterjrwrryrprqrorvrrr&rZcache_store_errorr[rr) rrZrobj_typeZ query_paramsrZobj_caseZwhere_pk field_nameZscopesZscopeZ model_nameZbindidr)rrrZrrrs   &              z CmfCache.addc Cs~zdtkrtt_Wn(tk r<}z WYdSd}~XYnXdtjkrTdtjd<ntjdd7<dtjkrzdtjd<dS)N profiler_data cache_hitr_ cache_missrrr:rrvrrZrrrrprofiler_data_hitZs    zCmfCache.profiler_data_hitc Cszdtkrtt_Wn(tk r<}z WYdSd}~XYnXdtjkrTdtjd<ntjdd7<dtjkrzdtjd<|tjd<dS)Nrrr_rrZcache_miss_lastrrrrrprofiler_data_misshs     zCmfCache.profiler_data_misscCstjdrdSt|jd||}|rdtkrtjr|d|tjfit|jd|dtj|jrt d|tjt|jd|t|jd || |t |S| |t|jd |dS) NrZCACHEr[rz*DEBUG_CACHE JSHASH: add invalidate jshash:rrzDEBUG_CACHE HIT: zDEBUG_CACHE GET:zDEBUG_CACHE MISS: )rrrRr4rCrrr[rrrrSrTrrrrrrRws"      z CmfCache.getcCs t|jdt|jdpdS)Nz%DEBUG_CACHE TRANS get_last_invalidatecache_last_invalidaterr4rCrpr=rRrGrrrrs zCmfCache.get_last_invalidatecCst|jd|t_dS)Nz)DEBUG_CACHE TRANS cache_transaction_start)r4rC gen_time_usrrrGrrrrs z CmfCache.cache_transaction_startcCsttjtjjdS)Nz%Y%m%d%H%M%S%f)rprinowrbrtstrftimerGrrrrszCmfCache.gen_time_uscCs:t|jd|s6|jdtj|jd|dS)NzDEBUG_CACHE TRANS cache_lock cache_lockr)r4rCrr=rrrrrGrrrrs zCmfCache.cache_lockcCsJdtkr dS|}tj|ks,tj|dkrFt|jd|jdddS)NrizDEBUG_CACHE TRANS cache_unlockr0)rrrr4rCr=r)rZlock_trrrr cache_unlocks  zCmfCache.cache_unlockcCs t|jdt|jdpdS)Nz%DEBUG_CACHE TRANS get cache_locked_byrrrrGrrrrs zCmfCache.cache_locked_bycCs"|}|dkr|tjkrdSdS)NrTF)rrr)rZ_cache_locked_byrrrrszCmfCache.cache_is_lockedcCsZg}|D]}|||q||||j||||jrVtd|d|j|dS)N invaldebug--)rrrrrCr)rrrrZinv_listrrrr_do_invalidates zCmfCache._do_invalidatec sd&fdd }ddlm}tjdr.dStjdd|j|j}ij rpddl }| |jd kr||j d d kr|d |j d |jd||j d|d|jdd|d|j d |jdd|dSt}|jddD]8\} } | jdkr&q t| tjjrΈdkrh| rh| j rh|d | j d |jddkr| jr| j| jkr|d | j jd |jd|d | j jd |jd|| q t| tjjrdkr| jr|| q | jr$t| tjjr$|| q | jr | j| jkr || q t|dkr|dkr|tjdd|j dSt|dr|jdkr||j ddkrb|D]} t|dr|d|jd |d | dt|d!r|d|j d |d | dt|drH|jD]$} |d| j d |d | d"q"|d|d | dqdkst|d#r|j jst|d$r|j!jst|dr|jjst|dr|jjrt|dr|d|jd |ddt|d!r|d|j d |ddt|d#r^|j jr^|j jr^|j j|j jkr^|d|j jd |dd%t|d$r|j!jr|j!jr|j!j|j!jkr|d|jjd |dd%t|dr|jD] } |d| j d |dd"q|d|d|dS)'Nrccs6|}|r2||<tjdd||dS)NzDEBUG_CACHE INVAL: r)rr4rC)ZinvaldebugZinvdrrrrr _check_invals z)CmfCache.invalidate.._check_invalr)rrzDEBUG_CACHE start invalidate rZ RelationCachezRelationCache.parent.id=)rrrrzref=zobj.id=rrzwhere=rTr)rcmf_modified_byrZ cmf_viewed_byrc)createrUrzref_new=zref_old=zDEBUG_CACHE ivalidate: z not fields_have_changesr)rrrUcache_cluster_idr parent_idz where_lst=parent cache_clusterz where_old=)rc)" collectionsrrrrRrr4rCrTRACEpdbZ set_tracerrr rwrxrjrkrrroldnewrrrrrrloadrrr) rrrrr rrrZ change_fieldsrrlstrr r invalidates      "          "   22  zCmfCache.invalidatecCs<|jD]\}}td|q|jD]\}}td|q$dS)NzDEBUG CmfCache.dumps)r;r)r<rx)rrZr}rrrr{Ks zCmfCache.dumps)NF)N)NN)N)N)-r'r%r2r;r<r=r>rCrrrHr^rarrrrrr staticmethodrorrrrWrrrrrrrrrrRrrrrrrrr rr{rrrrr5?sZ  @     6  )  r5cCsJt|}|D]4\}}t|trztj|||<WqYqXq|Sr)rrxrjror~rrrs)rZdctrrrrrCustomJSONDecoderQs rc@seZdZddZdS)CustomJSONEncodercCsznt|tjtjfr|j}t|tjs2t|tjrP|jsF|dWS|WSt|tj rd|jWSt |}Wnt k rYn Xt |St ||S)NZ)rjrZ CmfDateTimeZCmfDaternridateZtzinfoZ isoformatrliter TypeErrorrwr default)rriterablerrrr#]s   zCustomJSONEncoder.defaultN)r'r%r2r#rrrrr\sr)Z static_folderZ root_pathTZSEND_FILE_MAX_AGE_DEFAULT)SocketIO Namespace) RedisManager)Clientrcc CsdddDD]v}tj||}tj|s:tj||}ztj|sLWqWn"tt fk rptj YnXtjj |ddSt j D]$}||}|rtjj |ddSqtj dS)NcSsg|]}tjtj|qSr)rpathrrB PROJECT_DIR)rdrrrrszsend_static..)zcustom/static/zcommon/static/z cmf/static/zdist/cmf-angular/T)Z conditionalzdist/cmf-angular/index.html)flaskZhelpersZ safe_joinrr)isabsrisfiler" ValueErrorZ BadRequestZ send_fileAPPHOOK_STATIC_NOTFOUND)Z req_filenameZ directoryfilenamehookrrr send_statics   r4c@seZdZdZdejdddZddZedd Zed d Zed d Z ed dZ eddZ edddZ eddZ eddZdS)CmfRedisMemoryWrapperut Если есть self.redis - используется он Если нет - нужен фолбэк N)rediscCs||_i|_dSr)r6 memstorage)rr6rrrrszCmfRedisMemoryWrapper.__init__csfdd}|S)Ncs6|jrt|jj||Std|f||SdS)Nu&Redis memory wrapper запрещен!)r6rr' CmfAbortErrorrr*rrwrappersz/CmfRedisMemoryWrapper.fallback..wrapperr)r*r:rr9rfallbacks zCmfRedisMemoryWrapper.fallbackcOstdSrNotImplementedErrorrrrrrszCmfRedisMemoryWrapper.smemberscOstdSrr<rrrrrscCs |j|Srr7rR)rrZrrrrRszCmfRedisMemoryWrapper.getcCs|jdSr)r7clearrGrrrrFszCmfRedisMemoryWrapper.flushdbcCs$|j|d}||7}||j|<|S)Nrr>)rrZamountrnrrrr?s zCmfRedisMemoryWrapper.incrbyr_cCs |||Sr)r?)rrZr@rrrincrszCmfRedisMemoryWrapper.incrcGs$d}|D]}||jkr|d7}q|S)Nrr_r7)rnamesrnrrrrs   zCmfRedisMemoryWrapper.existscCs||j|<dSrrB)rrZrnrrrrszCmfRedisMemoryWrapper.set)N)r_)r'r%r2__doc__r6Redisrr;rrRrFr?rArrrrrrr5s&        r5c sfdd}tdzzttdttrtdttj}|jt t| D]Z}td||ddkrb|d  }t |g}|D]t j|t|d |d qqbW5QRXWn8ttfk rYn tk rtd YnXW5tdXqdS) Nc s~zFtdd|d|dt||W5QRWSQRXWn2tk rxtdd|d|dYnXdS)NzRun server event handler z(*z, **)z+_spawn_server_event_listener handler error )r) cmf_contextrvlogging exceptionrrZhandlerrr spawn_handlers "z3_spawn_server_event_listener..spawn_handlerz._spawn_server_event_listener: listen channels z Channels z*_spawn_server_event_listener: got message rymessagechanneldata)rRrPz"_spawn_server_event_listener error)timesleeprIr)_server_event_channelsREDIS_DBr6pubsubZ subscriberwZlistenrQrRgeventspawnjsonrT SystemExitKeyboardInterruptrvrJrK)rNrWrPrQZhandlersrrMr_spawn_server_event_listeners*      ,r]cCsttdSr)rXrYr]rrrrspawn_server_event_listenersr^csfdd}|S)Ncst||Sr)rUr)funcrQrrr0 sz"on_server_event..decoratorr)rQpatternr0rr`ron_server_events rbcache_settingsr#ryr6Z run_uwsgiu?При работе через uwsgi обязателен RedisrF)appZcors_allowed_originsloggerZengineio_loggerdbunix_socket_pathzunix://z?db=zflask-socketio)rQZ write_onlyclient_managerredis://host:6379/ message_queue)Payloadd)rez/custom/org_name)r=r>)r>u&Неизвестный тип кешаcCstjddr0dtjdddtd}ndtjdddtd}td ||d }|jtjGd d d |j}||_|S) Nr#rgzredis+socket://z?virtual_host=r_rirjrkZcrm)ZbackendZbrokercs8eZdZfddZd fdd ZddddZZS) z$make_celery_app..ContextTaskcs.tj||d|j|_d|j|_dS)uyФабрика все опции прописывает в классе, вызов конструктора пустойZcmf_task_only_once_queue__Zcmf_task_only_once_lock__N)rrronly_once_queue_flag_nameonly_once_lock_namerrrrrXsz-make_celery_app..ContextTask.__init__Ncst|ddrLtjj|jd|jp ddddd}|sLtd|jd dS|pRi}t|d drddntj j j |d <t j ||f|}|j tj j j |||jd }t|td ||S)N only_oncer_rrd<T)exZnxzSkip create task z - already started system_taskcmf_current_person_id)Ztask_idrrrr_zcmf-background-task-queued)rrVr6rrosoft_time_limitrr rcurrent_personrrnr apply_asyncrZr{socketiorW)rrrZ_my_custom_paramr.Z need_creater event_paramsrrrrx^s,   z0make_celery_app..ContextTask.apply_async)ruc _sddl}|dtddtjj|j|jp2ddd}|jj rzt d|jj t |ddrz| tj|jtd |jd d d zz>|rtjj|tjdt_|j||}td |jdd d Wnxtk rH}zXt d|jj td |jd|d d |jj r6t d|jj t!|W5d}~XYnXW5t |ddrz |WnJtjjk r}z&td |d|jd|jd d W5d}~XYnXX|jj rt d|jj t!|t"j#$|W5QRSQRXdS)NrrOF)init_views_and_dsrrtimeoutzcmf-background-task-start-rqz ContextTask z startT)flushzlock release error z , lock_name z , timeout rz successzcelery task error z error zcmf-background-task-error-zcmf-background-task-result-)%rSrTrIrVr6lockrprvrequestrryrWracquirerUror)rrelease exceptionsZ LockErrorr}r CmfPersonrRr0current_person_fieldsr_current_personrunrvrJrKrorDstdoutr~) rrurrrS redis_lockrresulterrorrrr__call__tsH       z-make_celery_app..ContextTask.__call__)NNN)r'r%r2rrxrr3rrrr ContextTaskWs r) rBrcrRrfrZconfrceleryZTask)Z redis_url celery_apprrrrmake_celery_appKs Fr)rqrtc Ks`tjD]T}tdz |Wqttfk r6Yqtk rXtd|YqXqdS)Nscheduler_minutelyuJОшибка выполнения периодической задачи )r0HOOK_CRON_MINUTELYr)r\r[rvrJrK)_kwargstaskrrrrs  rcCst|ttt|tr||}t|j |j |j td|_ d|_ tjdd|jd<d|jd<d|jd <|St}d |d <d |d <t||d<t|d<t|}d |_tjdd|jd<d|jd<d|jd <|SdS)N)coder description tracebackapplication/jsonOriginrAccess-Control-Allow-Origintrue Access-Control-Allow-CredentialsContent-Type, x-ijtAccess-Control-Allow-Headersirz Unknown errorrrr)rr r format_excrjrZ get_responserZr{rrrrR content_typerheadersrRr:roZjsonify status_code)rresponserrrhandle_exceptions4         rZjwt_rsaz jwt_rsa.pub)RSAZ cmf_deleted rg_member_ofZdefault_projectZemailZphone user_localZphone_internalZ primary_roleZ online_statusZvacation_startZ vacation_endZ on_vacationZactivity does_not_workZis_admin is_supportcCsPtjs dSztjs&tjjdtjdt_Wn tk rHt dYnXtjS)Nz.CmfPerson:00000000-0000-0000-0000-000000000001rrzapp.system_person() error) rB AUTH_ENABLEDr0 system_personrrrRrrvrJrKrGrrrrs  rc Cstjdrtjd}n^tjs$dSzttj}Wn tk rTt dYdSX|dkrbdSt |t ov|dd}|sdSt |}|jdsdSdt_t|jd dgd }|rt|d krdS|jd d }|sdStd|}r|\}}ndS|dks$|dks$|dkr,tj}nR|dksT|dksT|dksT|dkr\tj}n"|dksp|dkrxtj}ntj}t|tjjsdSd|kr|j|jd|ddgd} n|j|ddgd} | sdS|t| j krdt_!| t_"dS)u Проверяем, что пользователь пришел из share-ссылки Если идёт обращение к /files, то проверяем Referer Т.к. jsurl передаётся только в апи для других ендпоинтов не смысла путаться парсить body Вообще, для API тоже лучше использовать реферер, тогда не придётся по два раза парсить json Выставляем: - g.sharelink_access_request = True, если пользователь пришел по /share ссылке - g.sharelink_access_granted = True, если проверка hash прошла - g.sharelink_access_obj = объект, по которому проверяли ключ. Используется в API для доп.фильтрации /filesZRefererNu{_check_sharelink_access не смогли распарсить request.data для проверки прав доступаjsurlrcz/share/Trrr#rz*^(?P[^:-]*)[:-](?P.*)$docZDOC CmfDocumentZflZFLZIN_WORK CmfFolderCmfListZEPIr rgrsharelink_hashr)rr)#rr)r`rrRrRujsonrTrvrJrKrjr:rrsharelink_access_requestrZqueryrr&rematchgroupsrrrrrrkZ CmfEntityrrorsharelink_access_grantedsharelink_access_obj) rZjson_resr,Z hash_paramZobj_keyrrZobj_codeZ obj_modelrrrr_check_sharelink_accesss^   ( rcCs |t_tjS)uh!!! Эта функция не меняет контекст, только инициализирует.)rr)personrrrset_current_person]srcCsZt|dddkr|jStjs dSt|dr@t|dddk r@|jStdtjt j ddS)NrZFAIL_recursionz#Warning!!! Lazy Calc Current Personr@) rrrBrrrr)r print_stackrDrrGrrrrwcsrwc sztjstjjdgd}|jsztjjJddl }ddl |j |j d fddtdD|_|W5QRX|jt_Wn tk rtdYnXtjS) Napp_keyrrrcc3s|]}VqdSr)choice)rrZalphabetsecretsrr szapp_key.. zapp.app_key() error)r0rrCmfGlobalSettingsrRrkutilcmfutil disable_aclstringrZ ascii_lettersdigitsrrangesavervrJrK)rZsettingsrrrrrs    rc Cst|dddk r|jSztjjddtjd|_|jstjj ddd}| zPtjjddtjd|_|jst j jt j|j|_W5QRXtW5| X|jWStk rtdYnXdS) N_anonymous_userzanonymous@evateam.ruTrZinclude_deletedrzcreate-anonymous-user-lockrOr|zapp._anonymous_user() error)rrrrrRr0rrVr6rrrrkrrr system_dataZcreate_anonymous_userrdd commit_all_dsrvrJrKrrrrranonymous_users4   rc Cst|dddk r|jSztjjddtjd|_|jstjj ddd}| zPtjjddtjd|_|jst j jt j|j|_W5QRXtW5| X|jWStk rtdYnXdS) N_sharelink_anonymous_userzsharelink-anonymous@evateam.ruTrz$create-sharelink-anonymous-user-lockrOr|z%app._sharelink_anonymous_user() error)rrrrrRr0rrVr6rrrrkrrrrZcreate_sharelink_anonymous_userrrrrvrJrKrrrrsharelink_anonymous_users8   rc Cst|dddk r|jSztjjddtjd|_|jstjj ddd}| zPtjjddtjd|_|jst j jt j|j|_W5QRXtW5| X|jWStdYnXdS) N_test_guest_userztest-guest@evateam.ruTrzcreate-test-guest-user-lockrOr|zapp._test_guest_user() error)rrrrrRr0rrVr6rrrrkrrrrZcreate_test_guest_userrrrrJrKrrrrtest_guest_user s4   rc Cst|dr t|dddk r |jStjjdgd|_|jstjj 8ddl m }|tjjdgd|_|jsxt dW5QRX|jS)N_global_settingsrrr)init_global_settingsu6CmfGlobalSettings не инициализирован.) rrrrrrRrkrrrcmf.system_datarAssertionError)rrrrrglobal_settings3s rcCs8t|dr t|dddk r |jStjjdgd|_|jS)N _global_varrr)rrrrZ CmfGlobalVarrRrGrrr global_varEsrlinenoc Cs6|tddtddf}|j|dd}td|t|d|dD]\}}|jd}tj |j tjd d}td |||j |j d ft|j |j }|rtd ||jD] } t| qqJ||d} | rtd d| D} tdt| | d ftdd|D} td| d dS)NFzz T)Z cumulativez Top %s linesr_rz#%s: %s:%s: %.1f KiBiz %scss|] }|jVqdSrsizerstatrrrrhszdisplay_top..z%s other: %.1f KiBcss|] }|jVqdSrrrrrrrjszTotal allocated size: %.1f KiB)Z filter_traces tracemallocZFilterZ statisticsr) enumeraterrseprr2r&rr linecachegetlinestripformatsumr) ZsnapshotZkey_typelimitZ top_statsr"rframer2lineZllotherrtotalrrr display_topOs.       rcCsJttj}ttjd|}tjdd|jd<d|jd<d|jd<|S) Nz/auth/signin?next_url=rrrrrrr)rrr,ZredirectrBAUTH_SERVER_URLrrR)Znext_urlrDrrr auth_redirectns    rc Cstjjjdddtjj|jtj d}|sdd|j ddD}t j |d <t j |d <t j |d <tjf|}|rd|_ |jtjd|_|jtjtjj|jdd W5QRXt|W5QRSQRXdS) Nzcreate-current_person-lock r|)loginrcSs(i|] \}}|ds|dkr||qS)Zcmf_r)r`)rrrrrr }s  z+create_person_from_auth..TrZ cmf_ownerZ cmf_authorr)Z only_data)rkrrZCmfLockrrrRrr0rrxrrrrrZCmfPersonGroupZ support_grouprZ user_grouprrrr)authrrZ person_valuesrrrcreate_person_from_authws&     rc sttjdkrdStdt_it_gt_ t j ddkt_ ddlmddlm}t_tjt_d |d d d t_dt_gt_gt_dt_dt_dt_dt_dt_dt_dt_dt_dt_ dt_!dt_"dt_#dt_$dt_%t&j&'t&j(j)t_*d dt_'tj*+t_,ddt_+dt_-it_.it_/gt_0dt_1dt_2fdd}|t_3t4j5sjt6dt7j89dStj:;dr|dSd}tj:;dr0tj< dptj< dptj= d}|r0|;dr|t>dd}t7j?@|}t7j?j |dtAjBd}t3d|dd d|d||s0tCddd d!Stj:d"krLt7jDjEtAjBd#}|stjF d$}|rt7jGH|t_t4jIot7jGj tjjjJd%}t4jIrtjr|sdt_tjrtjjjJt_t7j?j dtjjtAjBd&}|r|jKrd}n(|stjjLrtM|ptjtjjLd'}tj:;d(p8tj:;d)p8tj:;d*}d+d,}|rLd}|r|jjJt_tjrtjjLt_tjjNt_tjjN t_ndt_dt_dt_dt_t6|n`|s|rtjO} tPtj$rtjQ} dt_dt_dt_dt_t6| ndt_dt_t6tjRtj ptj:;d-ptj:d.kptj:;d/ptj:;drdtjS d0d1krdtjTd2kptj:;d3ptj:d4kptj:;d5ptj:;d6rtjTd2kptj:;d7rtjTd2kp| rtj:;d8rt7jUVptj:;d9otj} d} | sztjsztjr,|s,tWd:tjd;dd<tjrdd d!} ntX} t7j89tjR} tjY} t7j8Z| S)?uJ Должны вернуть Response or None Особенности поведения: В случае web запроса: - response - view не выполняется, сразу отправляем response - None - выполняем view - Exception(в т.ч. и abort) - обработчик выводит трейс клиенту Системный контекст(shell, celery, socket, other with cmf_context): - None, Response - нет разницы - Exception - прерывает выполнение. ZOPTIONSNFZ run_pytest1rrS)choicesrcZ0123456789abcdef)rcSstjSr)r_nowrrrrz before_request..cSstjSr)r_daterrrrrrc sttddrڈ}tt|tjdd}tt|tjdd}tdidt}|dd}|d d}|d d}t j d }t tj d |d |d |dd |d|d|d|||dd f|t_dS)NrCFrIrdrZ select_countZselectrrinsertmr#z(+z)mszs zu zi t) rrBroroundr debug_start debug_nowrRr:rrr) request_id) messagesrrrEZscZscsZscuZscirrrrr s,     "  zbefore_request..debugz/memZOKz/api/z X-Eva-TokenZ AuthorizationtokenzBearer T)api_token_hashrrzAuth API Token(z ...): hash=z , person=zInvalid API tokeni text/plain)rz /pub/webhookr access_token)r)rrr)rz/pub/apiz/docsz/sharecSstjdptjdS)Nr /socket.io/)rr)r`rrrris_allow_publicAs  z'before_request..is_allow_publicz /auth/sync)z /auth/invitez/auth/restore_passwordz /auth/signmoderootz 127.0.0.1z /crm/forms/z/manifest.webmanifestz /app/assets/z/forms/z/js/z/pub/z /helpdeskuУчётная запись uQ заблокирована. Обратитесь к администратору)r rznot authorized)[rrrmethodrbefore_requestrrflags remove_filesrrrRZtestrSZrandomrr rrr Zsave_only_data_hackalertdeferred_tasksrZ public_accessrZ is_local_userZ is_guest_userZ is_anonymousrZis_system_contextZdisable_notifyZ import_modeZinteractive_shellrrrrirrbrtrr rZrelaxed_bz_python_mode emit_list socket_events server_eventsrZjscache_timelifer rBrrr CmfAccessListZ init_contextr)r`rrrrZ hash_tokenr0rr Z CmfWebhookZget_token_personZcookiesZCmfAuthZfrom_jwtZIS_AUTHORIZATORrnrZjwt_is_supportrZjwt_is_match_orgrrrrrZ remote_addrZ CmfPluginZ check_secretrrrZ setup_context) rr Z auth_personZ api_tokenr rZ real_authZis_force_publicrZ anon_personZis_allow_without_authrrrrrrs@            $                     rcCstttdSr)rr emit_eventsemit_server_eventsrrrrcommit_with_eventsr c CsNtdsdStjD]*\}}td|d|d|dddqit_dS)Nr is_changed-rr#rprivate) namespacer)rrRrrxr)rr}rrremit_list_applys  $r$c Cs|D]\}}|D]}|d}|d=t|d}|d=|d}z"t|t|t|WnJtk r}z,td|d|d|d|WYqW5d}~XYnXt|ts|j |f|d|d |d iq|d r,|d d ks,|d |d <|j |f|d|d |d iq| d r|drd |ddkrj| |ddd d|ddkr|dddD]}| |q|D]r} t| t j jrtd| d|d|d|| jj} nt| } | |d <|j |f|d|d |d iqqqdS)N socket_clientrruPDEV: FATAL emit пришел с несериализуемыми данными z! socket_client:z; event_persons:z; event_param:rr#rr"rZevent_current_personrZrelation_personsuYDEV: INFO Кривой евент, в event_person попал объект, а не str: r)rxrcopyrZr{rvr)rjr%rWr`rrkrrrrnro) Z event_dictmsgZevent_param_listZ event_paramr%rZ emit_kwargsrZuser_id event_personevent_persons_strrrremit_events_dictsF       " "    r*cCs&ttdpi}it_t|dS)Nr)r$rrRrr*)rrrrrsrcs:fdd}ttdp iit_r6t|dS)uЗапускаемся в контексте, но отправляем в отдельном таске вне контекста, после паузы, чтобы не держать запрос.csttdSr)rXrTr*rdelayrrremitters z'emit_events_with_delay..emitterrN)r$rrRrrXrY)r,r-rr+remit_events_with_delays r.cCsPtdp g}gt_|D]2\}}td|d|tj|t|qdS)Nrzemit_server_events: r) rrRrr rVr6ZpublishrZr{)rrQrRrrrrs  rcCstit_dSr)rZrollback_all_dsrrrrrrrollback_purge_eventsr/cCs$tdgD] }|}q gt_dS)Nr)rrRrxr)Ztask_signaturerrrrapply_deferred_run_taskss r0cCs2tdgD]}tj|r t|q gt_dS)Nr)rrRrr)rrrr2rrrapply_deferred_remove_filess  r2cCsNttdttttjdd|jd<d|jd<d|jd<|S) uk Запускается только если небыло ошибок :param r: :return: rdrrrrrrr) rrr.rr0r2rrrR)rDrrracaos  r3c CsTtjs dStjt_t,tjj tj j j W5QRXW5QRXtdSr)rBrZDISABLE_PERMISSIONSr0disable_permissionsrIrkrrrZincluderrZ load_acl_datar^rrrr app_init_acl,s "r5cCs*tttjrttjr&tdSr)rbefore_first_requestr5rBWHATSAPP_MESSENGER_URLspawn_messenger_socketio_clientWHATSAPP_WEB_SOCKETspawn_whatsapp_socketio_clientrrrrr68s r6cCstj|ddSNrK)rrr<rrrteardown_requestDsr=c Cstd|d|d|d||r0tjs0dS|dk rH|tj krHdS|rrdt}td|t||tj krtj |dS)u Глобальная функция для оповещения фронта о предупреждениях :param msg: :return: zcmf_alert: msg=z abort=z debug_only=z devel_only=Nrzcmf_alert Stack: ) rr rBrCZ PRODUCTIONrr format_stackr8rr)r'r rZ devel_onlyZ stack_strrrrrJs"  rr#r")r#r%rrc OsZ|st}zdtkrtt_Wn(tk rD}z WYdSd}~XYnXt|dkrldtjrdtjjjpfdig}t||||d}t |||d<d} dD]} | | rd} qq| rt d t |stjg}| std t g} |D]2} t| tjjr| | jjq| t| q| |d <t |d tj|g} || krV| |dS) NrrrM)rrr#rr%F) zevent-rz task-comment-zaudit-task-comment-znotify-person-r!zcomment-z notify-osznotify-important-ZDEBUG_TuCDEV: WARNING в cmf_emit_event не указали cmf_emit_event: uDEV: FATAL!!!!!!!!!!!!!! Добавьте меня в спинт Баги реактивности. 10.10.2022 в cmf_emit_event не указали cmf_emit_event: r)ryrr:rrvrrwrrnrZr{r`r)localsrrjrkrrrro setdefault)r'r#r%rrrrrrzZmutedZ muted_debtr)r(Z event_listrrrrbsB     rcCs"dtkrgt_tj||fdS)Nr)rrr)rQrRrrrcmf_emit_server_eventsrAcCs|rd|nd}|rdnd}|r&dnd} d|||| g} tj| dtjtjd} |r| \} } |rv| } | } | j}|r|rtd| td | td | t d | |r|| | fS|| p| fSd S) u :param command: собственно, команда которую необходимо выполнить :param sudo: необходимо ли команду выполнять от рута :param timeout: секунд на выполнение. Дальше команда получит SIGTERM. :param wait: True дождаться окончания выполнения команды, False запустить и забыть, вернёт (None, None) :param separate_out_and_err: отдельно stdout, отдельно stderr, иначе не ясно как это всё парсить False по умолчанию, чтобы никому ничего не сломать. :param do_decode: возвращать декодированные строки вместо байт :param do_raise: выбрасывать исключение в случае, если код возврата != 0 (удобно для логирования трэйсбэков) TODO: почему u''? Наследие python2 и биллинга или реальная необходимость? TODO: Можно перейти на f'{sudo}bash -c export BLA=BLA;{timeout}{cmd}{redirect}', читается лучше z timeout {0} rczsudo z 2>&1z0{0} bash -c "export BOOTUP="noncolor";{1}{2}{3}"T)shellrrEu6Произошла ошибка при вызове %sz STDOUT: %sz STDERR: %su4Произошла ошибка при вызове )NN) r subprocessPopenPIPEZ communicaterQ returncoderJr RuntimeError)ZcommandZsudor}waitZseparate_out_and_errZ do_decodeZdo_raiseZ timeout_cmdZ_sudoZ _redirectcmdZprocessouterrZerrcoderrrrun_bash_commands8        rLcCsLttj}t}||||}t | }|d|}|S)ua Подписываем токен сертификатом :param jwt: :return: r ) r rr0rsa_private_keyr rrsignbase64 b64encoderQ)jwtZsignerZdigestrNrrrrrsa_sign_pack_jwts  rRcCsHddd}tt|}tt|}|d|S)uD Создаем токен :param payload: :return: ZRS256ZJWT)Zalgtypr )rOrPrZr{rrQ)payloadheaderrrr create_jwts rVcCstjS)u2 Текущая версия :return: )rBZ CMF_VERSIONrrrrcmf_get_versionsrWcCs|tjkrtj|dS)ux Удаления файлов после коммита транзакции :param filename: :return: N)rrrr1rrrcmf_remove_files rXc@s,eZdZd ddddZddZdd ZdS) rIFNr6cCstd|_||_dS)Nz /with-context)r0Ztest_request_contextctxr{)rr{rrrrs zcmf_context.__init__cCs*|j|jrtttdSr)rY __enter__r{ start_viewsrZinit_dsr0Zpreprocess_requestrGrrrrZs  zcmf_context.__enter__cCs<tj|d|s(tttt|j|||dSr;)rrrrr0r2rY__exit__)rexcrnrrrrr\s zcmf_context.__exit__)F)r'r%r2rrZr\rrrrrIsrIcCsddddddg}tjdd|D}|D]\}}}|tjd }|d r(|D]\}|d sf|d rN|d sNddd|dD}t j |d|ddddqNq(t j ddt dt j ddt dt j ddt ddS)Nz cmf/viewsz cmf/modulescommonmodulesz custom/commonzcustom/modulescSs&g|]}tjtjd|ddqS)r#T) followlinks)rwalkrBr*)rZ search_dirrrrr szstart_views..rcz/viewsz.pyz.pycz __init__.pyr css|]}|r|VqdSrr)rrrrrr szstart_views..r#r)packager)rZ view_funcz/Zdistz/static/Zstatic) itertoolschainrurBr*endswithr`rr& importlib import_moduler0r(r4)Z search_dirsZ search_iterdirnamedirnames filenamesr2r+rrrr[ s$  $r[cKs,tj|||d}|jf|tj||S)NrL)r signaturerrrr)rrrr.rkrrrcmf_deferred_task" s  rlcCs*tjdddkr&ttjdddgdS)NSOCKETIOZworkerz--poolrX)rrrRrXrYrZ worker_mainrrrrspawn_task_worker) srncCs*dd}tjdddkr&td|dS)Nc SsHddlm}t|tdW5QRXtdtjdtddS)NrBeat)rdzBeat die!!! Stop worker.r@r_) Zcelery.apps.beatrprIrrr)rDrEexitrorrrrun_beat1 s  z&spawn_task_scheduler..run_beatrmrO)rrrRrXZ spawn_later)rrrrrspawn_task_scheduler0 srscCs"dd}tjrtj|dgddS)NcSs2i}tjddrdtji}tjtj||ddS)Nrmr namespacesr)rrrRrBORG_NAME whatsapp_sioconnectr7rtrrrmessenger_connect= s z:spawn_messenger_socketio_client..messenger_connectz /whatsapp)ru)rBr7rXrY)ryrrrr8< sr8cCsttjtjdSr)rXrYwhatsapp_go_siorxrBr9rrrrr:H sr:cCs<ddl}tdtjdtj|jtjtjdddddS)NruYПодключаемся к удаленной отладке config.REMOTE_DEBUG_SERVER=z config.REMOTE_DEBUG_PORT=TF)ZportZstdoutToServerZstderrToServerZsuspend)pydevd_pycharmr)rBZREMOTE_DEBUG_SERVERZREMOTE_DEBUG_PORTsettrace)r{rrrcmf_pycharm_debugL s r}r6cCsHttjtjttttttdtjj dd}t |}t |S)NrrZguid)ZissZorg_nameZiatZexprZjti) r:rBZAPP_FQDNrvrorprSrrrnrVrR)rTrQrrrgen_eva_app_tokenS sr~)urirRcCsddlm}|}dddd|_tj|}t|d<|j|t|d}|j d krtt d |d |j d |j |}|rd |krt d|t ||S)Nr)sessionrrr)z Content-typeZAcceptzContent-EncodingZ eva_app_token)rRu9Не удалось отправить запрос, url=z, status_code=z, res=ru(Сервер вернул ошибку: )ZrequestsrrrBrr~ZpostrZr{rrvZcontentrr )rrRrrr,rrrrrcall_eva_account_ s$     r)rc)NN)rr)F)r)N)FFN)TNTFFF)NN)rrcrSrDrOZcalendarrirCrSrrrrpathlibrZ urllib.parserrrrrr~rXrZ Crypto.Hashr ZCrypto.Signaturer r,r r Z flask.jsonr Zwerkzeug.exceptionsrrkrZ cmf.includeZcmf.data_providersrrrrr4objectr5rrr'rBr*r0Z json_encoderrrr1rZHOOK_CRON_HOURLYZHOOK_CRON_DAILYZHOOK_CRON_WEEKLYZHOOK_CRON_MONTHLYr4Zflask_socketior%r&ryr'r(r6r4r5rVrwrUr]r^rbrrcr&Zcfgr$rGrrrRrvrCZsocketio_kwargsrfr,rhrlZengineio.payloadrmZmax_decode_packetsrwrzr)ropenr*readrrvrrrrrZ errorhandlerrZregister_error_handlerZprivate_key_pathZpublic_key_pathZCrypto.PublicKeyrZ import_keyZ read_bytesrMZrsa_public_keyrrrrwrrrpropertyryrZ current_userrrrrrrr r$r*rr.rr/r0r2Z after_requestr3r5r6r=rrrArLrRrVrWrXrIr[rlrnrsr8r:r}ror~r:rrrrrs~              ;'            V   !    bV    1       1 ,