微服务全链路跟踪:jaeger集成istio,并兼容uber-trace-id与b3
当springcloud服务集成hystrix,并且用了hystrixCommend注解到方法上时,jaeger链路会断掉
在网上搜索到了大量jaeger遇到多线程时的处理方式,都是包装线程池来做到ThreadLocal传递,有很多都用到了阿里开源的transmittable-thread-local。
下面说一下当集成hystrix时,jaeger链路丢失问题,大家都知道hystrix默认是线程池隔离,所以归根结底还是遇到多线程线程变量没有共享的问题,网上也罗列了几种方案:
hystrix.command.default.execution.isolation.strategy: SEMAPHORE
当并发高时这里设置信号量隔离是有风险的,可以根据情况优化断路器配置来降低风险
隔离策略官方文档有定义:
原先我就定义了一个feign传递request中header信息的策略,在原有的隔离策略下面参考https://github.com/opentracing-contrib/java-concurrent项目的代码:
将原有feign自定义隔离策略做了响应变动,代码如下
/**
* 自定义Feign的隔离策略:
* 在转发Feign的请求头的时候, 如果开启了Hystrix,
* Hystrix的默认隔离策略是Thread(线程隔离策略), 因此转发拦截器内是无法获取到请求的请求头信息的,
* 可以修改默认隔离策略为信号量模式:hystrix.command.default.execution.isolation.strategy=SEMAPHORE,
* 这样的话转发线程和请求线程实际上是一个线程, 这并不是最好的解决方法, 信号量模式也不是官方最为推荐的隔离策略;
* 另一个解决方法就是自定义Hystrix的隔离策略:
* 思路是将现有的并发策略作为新并发策略的成员变量,在新并发策略中,
* 返回现有并发策略的线程池、Queue;将策略加到Spring容器即可;
*/
@Component
@Slf4j
public class FeignHystrixConcurrencyStrategy extends HystrixConcurrencyStrategy {
private HystrixConcurrencyStrategy delegate;
@Lazy
@Autowired
private Tracer tracer;
public FeignHystrixConcurrencyStrategy() {
try {
this.delegate = HystrixPlugins.getInstance().getConcurrencyStrategy();
if (this.delegate instanceof FeignHystrixConcurrencyStrategy) {
// Welcome to singleton hell...
return;
}
HystrixCommandExecutionHook commandExecutionHook =
HystrixPlugins.getInstance().getCommandExecutionHook();
HystrixEventNotifier eventNotifier = HystrixPlugins.getInstance().getEventNotifier();
HystrixMetricsPublisher metricsPublisher = HystrixPlugins.getInstance().getMetricsPublisher();
HystrixPropertiesStrategy propertiesStrategy =
HystrixPlugins.getInstance().getPropertiesStrategy();
this.logCurrentStateOfHystrixPlugins(eventNotifier, metricsPublisher, propertiesStrategy);
HystrixPlugins.reset();
HystrixPlugins instance = HystrixPlugins.getInstance();
instance.registerConcurrencyStrategy(this);
instance.registerCommandExecutionHook(commandExecutionHook);
instance.registerEventNotifier(eventNotifier);
instance.registerMetricsPublisher(metricsPublisher);
instance.registerPropertiesStrategy(propertiesStrategy);
} catch (Exception e) {
log.error("Failed to register Sleuth Hystrix Concurrency Strategy", e);
}
}
private void logCurrentStateOfHystrixPlugins(HystrixEventNotifier eventNotifier,
HystrixMetricsPublisher metricsPublisher,
HystrixPropertiesStrategy propertiesStrategy) {
if (log.isDebugEnabled()) {
log.debug("Current Hystrix plugins configuration is [" + "concurrencyStrategy ["
+ this.delegate + "]," + "eventNotifier [" + eventNotifier + "]," + "metricPublisher ["
+ metricsPublisher + "]," + "propertiesStrategy [" + propertiesStrategy + "]," + "]");
log.debug("Registering Sleuth Hystrix Concurrency Strategy.");
}
}
@Override
public <t> Callable<t> wrapCallable(Callable<t> callable) {
RequestAttributes requestAttributes = RequestContextHolder.getRequestAttributes();
return new WrappedCallable<>(callable, requestAttributes,tracer,tracer.activeSpan());
}
@Override
public ThreadPoolExecutor getThreadPool(HystrixThreadPoolKey threadPoolKey,
HystrixProperty<integer> corePoolSize,
HystrixProperty<integer> maximumPoolSize,
HystrixProperty<integer> keepAliveTime,
TimeUnit unit, BlockingQueue<runnable> workQueue) {
return this.delegate.getThreadPool(threadPoolKey, corePoolSize, maximumPoolSize, keepAliveTime,
unit, workQueue);
}
@Override
public ThreadPoolExecutor getThreadPool(HystrixThreadPoolKey threadPoolKey,
HystrixThreadPoolProperties threadPoolProperties) {
return this.delegate.getThreadPool(threadPoolKey, threadPoolProperties);
}
@Override
public BlockingQueue<runnable> getBlockingQueue(int maxQueueSize) {
return this.delegate.getBlockingQueue(maxQueueSize);
}
@Override
public <t> HystrixRequestVariable<t> getRequestVariable(HystrixRequestVariableLifecycle<t> rv) {
return this.delegate.getRequestVariable(rv);
}
static class WrappedCallable<t> implements Callable<t> {
private final Callable<t> target;
private final RequestAttributes requestAttributes;
private final Span span;
private final Tracer tracer;
public WrappedCallable(Callable<t> target, RequestAttributes requestAttributes,Tracer tracer, Span span) {
this.target = target;
this.requestAttributes = requestAttributes;
this.tracer=tracer;
this.span=span;
}
@Override
public T call() throws Exception {
Scope scope = span == null ? null : tracer.scopeManager().activate(span);
try {
RequestContextHolder.setRequestAttributes(requestAttributes);
return target.call();
} finally {
RequestContextHolder.resetRequestAttributes();
if (scope != null) {
scope.close();
}
}
}
}
}
其中改动部分就是在原有的WrappedCallable中增加了span、trace的注入。
至于自定义隔离策略以及Callable是可以支持多个链的,这里不做详细描述,大家有兴趣可以参考,下面的链接:
https://blog.csdn.net/songhaifengshuaige/article/details/80345012