我正在尝试构建GCP云功能,它是通过云调度器触发的,用于收集来自博彩公司API的赔率数据,用于各种不同的运动和比赛,按不同的时间表进行。
我正在寻求一些关于我的方法的一些建议,关于这是否是正确的策略,或者是否有更好的方法来实现我想要的。
我一直在使用python作为代码和GCP函数(Gen 2)、、Pub/Sub。火药恢复。
目标:使用从博彩公司收集赔率数据,并在Firestore中存储所需的数据点。
计划:安装云功能(Gen 2)和Cloud一起收集每一项运动比赛的赔率数据,这些比赛按不同的时间表进行。
问题:每项运动将有3-5个博彩市场为每次活动收集.有时,庄家API提要要求您发出一个API调用来获取事件Ids,然后使用API URL中的那些事件ids进行另一个调用来获取事件概率数据。因此,有3-5个API调用来收集每个游戏所需的所有博彩市场数据。
目前,我将这些不同的API调用设置为不同的函数,当我在本地运行它时,它似乎可以正常工作。
方法:我想使用一对多的PubSub消息传递系统在云函数(Gen 2)上设置每个函数。
我需要函数1首先运行(云调度程序),收集数据并推送到pubsub,然后,一旦消息在pubsub中,我就需要函数1和2来运行消息,并将其拉到函数中。
这是函数1的代码,它收集了游戏的事件id:
import requests
import json
from firebase_admin import firestore
from google.cloud import pubsub_v1
db = firestore.Client(project='xxxxxxx')
# API INFO
Base_url = 'https://xxxxxxx.net/v1/feeds/sportsbookv2'
Sport_id = '1000093204'
AppID = 'xxxxxxx'
AppKey = 'xxxxxxx'
Country = 'en_AU'
Site = 'xxxxxxx'
project_id = "xxxxxxx"
topic_id = "xxxxxxx_basketball_nba"
publisher = pubsub_v1.PublisherClient()
# The `topic_path` method creates a fully qualified identifier
# in the form `projects/{project_id}/topics/{topic_id}`
topic_path = publisher.topic_path(project_id, topic_id)
def win_odds(request):
event_ids = []
url = f"{Base_url}/betoffer/event/{','.join(map(str, event_ids))}.json?app_id={AppID}&app_key={AppKey}&local={Country}&site={Site}"
print(url)
windata = requests.get(url).text
windata = json.loads(windata)
for odds_data in windata['betOffers']:
if odds_data['betOfferType']['name'] == 'Head to Head' and 'MAIN' in odds_data['tags']:
event_id = odds_data['eventId']
home_team = odds_data['outcomes'][0]['participant']
home_team_win_odds = odds_data['outcomes'][0]['odds']
away_team = odds_data['outcomes'][1]['participant']
away_team_win_odds = odds_data['outcomes'][1]['odds']
print(f'{event_id} {home_team} {home_team_win_odds} {away_team} {away_team_win_odds}')
event_ids.append(event_id)
# WRITE TO FIRESTORE
doc_ref = db.collection(u'xxxxxxx_au').document(u'basketball_nba').collection(u'win_odds').document(
f'{event_id}')
doc_ref.set({
u'event_id': event_id,
u'home_team': home_team,
u'home_team_win_odds': home_team_win_odds,
u'away_team': away_team,
u'away_team_win_odds': away_team_win_odds,
u'timestamp': firestore.SERVER_TIMESTAMP,
})
return event_ids
print(f"Published messages to {topic_path}.")
下面是函数的代码,这些函数将从pubsub中提取事件is,并在其API调用中使用这些函数来收集每个博彩市场上的数据。
下面是订阅者函数的模板,它将从pubsub订阅中提取事件in消息,然后在API调用中使用该函数的那些事件in。
from concurrent.futures import TimeoutError
from google.cloud import pubsub_v1
import requests
import json
from firebase_admin import firestore
import functions_framework
db = firestore.Client(project='xxxxxxx')
# API INFO
Base_url = 'https://xxxxxxx.net/v1/feeds/sportsbookv2'
Sport_id = '1000093204'
AppID = 'xxxxxxx'
AppKey = 'xxxxxxx'
Country = 'en_AU'
Site = 'xxxxxxx'
project_id = "xxxxxxx"
subscription_id = "xxxxxxx_basketball_nba_events"
timeout = 5.0
subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(project_id, subscription_id)
@functions_framework.cloud_event
def win_odds(message):
print(message.data)
event_ids = json.loads(message.data)
url = f"{Base_url}/betoffer/event/{','.join(map(str, event_ids))}.json?app_id={AppID}&app_key={AppKey}&local={Country}&site={Site}"
print(url)
windata = requests.get(url).text
windata = json.loads(windata)
for odds_data in windata['betOffers']:
if odds_data['betOfferType']['name'] == 'Head to Head' and 'MAIN' in odds_data['tags']:
event_id = odds_data['eventId']
home_team = odds_data['outcomes'][0]['participant']
home_team_win_odds = odds_data['outcomes'][0]['odds']
away_team = odds_data['outcomes'][1]['participant']
away_team_win_odds = odds_data['outcomes'][1]['odds']
print(f'{event_id} {home_team} {home_team_win_odds} {away_team} {away_team_win_odds}')
# WRITE TO FIRESTORE
doc_ref = db.collection(u'xxxxxxx_au').document(u'basketball_nba').collection(u'win_odds').document(
f'{event_id}')
doc_ref.set({
u'event_id': event_id,
u'home_team': home_team,
u'home_team_win_odds': home_team_win_odds,
u'away_team': away_team,
u'away_team_win_odds': away_team_win_odds,
u'timestamp': firestore.SERVER_TIMESTAMP,
})
if __name__ == '__main__':
streaming_pull_future = subscriber.subscribe(subscription_path, callback=win_odds)
print(f"Listening for messages on {subscription_path}..\n")
# Wrap subscriber in a 'with' block to automatically call close() when done.
with subscriber:
try:
# When `timeout` is not set, result() will block indefinitely,
# unless an exception is encountered first.
streaming_pull_future.result(timeout=timeout)
except TimeoutError:
streaming_pull_future.cancel() # Trigger the shutdown.
streaming_pull_future.result() # Block until the shutdown is complete.
print("Listening, then shutting down.")
发布于 2022-11-11 12:36:27
https://stackoverflow.com/questions/74369077
复制相似问题