Thingsboard AWS服务器冻结
我想在AWS (t2.microl)上运行一个自管理的Thingsboard。
我已经在t2.微AWS实例上安装了Thingsboard CE，运行Ubuntu20.04服务器。
我遵循aws设置和Ubuntu安装指南(postgresql+内置队列服务)。
我还使用这指南建立了haproxy。
我成功地登录到了我的东西板。我只更改了密码，检查了基本功能，但没有创建任何新的仪表板，也没有做任何修改。
在这之后，我开着电脑，运行着Thingsboard。第二天，我无法到达Thingsboard，尽管AWS实例正在运行，但我不能再深入到它了。在停止和启动(重新启动没有工作)之后，实例一切正常(可以使用ssh，可以访问Thingsboard )。
我只需保持实例就可以重现这个失败，似乎在服务器时间(5-8小时)之后，Thingsboard(或其他一些不确定的东西)出现了故障，冻结了整个计算机。
我检查了两件事：
- 我检查了AWS监控中CPU的使用情况。似乎过了几个小时，CPU负载有了很大的提高，然后又回落到了几乎为零的水平。当Thingsboard运行时，它是常量的。参见AWS监视的printscreen
- 我检查了Thingsboard日志(在/var/log/thingsboard中)：有一些错误，但不幸的是，大多数事情都不足以让我猜测新安装的问题是什么。下面是日志中的一些行：
2021-11-12 00:21:59,626 http-nio-0.0.0.0-8080-exec-13 INFO o.a.coyote.http11.Http11Processor -错误解析HTTP请求标头注意: HTTP请求解析错误的进一步发生将在调试级别记录。java.lang.IllegalArgumentException:在方法名中找到的无效字符 0x160x030x010x00{0x010x000x00w0x030x030x170xb80xb80xe50xef0x000xb50x0a&0x930x020x00:0xde0xd70xa00xab0xb 70x8bU0xc00x92r0x9330x10O0x8cHTTP方法名必须是org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:417) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:261) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor上的标记org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)，java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)，java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)，org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)，.doRun(NioEndpoint.java:1707)在java.base/java.lang.Thread.run(Thread.java:829) 2021-11-12 00:22:01,486 sql-queue-2-ts-4-线程-1警告com.zaxxer.hikari.pool.PoolBase - HikariPool-1 -验证失败 connection org.postgresql.jdbc.PgConnection@4393afd0 (此连接已关闭)。可能考虑使用较短的maxLifetime值。2021-11-12 00:22:01,487 sql-queue-2-ts最新的8-线程-1警告com.zaxxer.hikari.pool.PoolBase - HikariPool-1 -未能验证连接org.postgresql.jdbc.PgConnection@75b9496b (此连接已关闭)。可能考虑使用较短的maxLifetime值。2021-11-12 00:22:01,487 sql-queue-0-ts最新的-6-线程-1警告com.zaxxer.hikari.pool.PoolBase - HikariPool-1 -未能验证连接org.postgresql.jdbc.PgConnection@31849eec (此连接已关闭)。可能考虑使用较短的maxLifetime值。2021-11-12 00:22:01,487 sql-queue-0-ts-2-线程-1警告com.zaxxer.hikari.pool.PoolBase - HikariPool-1 -未能验证连接org.postgresql.jdbc.PgConnection@725fafe3 (此连接已关闭)。可能考虑使用较短的maxLifetime值。
更多的：
2021-11-12 00:23:46,205 [sql-log-1-thread-1] INFO o.t.s.dao.sql.TbSqlBlockingQueue - Queue-2 [TS Latest] queueSize [9] totalAdded [0] totalSaved [0] totalFailed [0]
2021-11-12 00:23:47,741 [sql-queue-0-ts-2-thread-1] WARN o.h.e.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: 08003
2021-11-12 00:23:47,742 [sql-queue-2-ts-4-thread-1] WARN o.h.e.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: 08003
2021-11-12 00:23:47,742 [sql-queue-2-ts latest-8-thread-1] WARN o.h.e.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: 08003
2021-11-12 00:23:47,742 [sql-queue-0-ts latest-6-thread-1] WARN o.h.e.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: 08003
2021-11-12 00:23:48,022 [sql-queue-0-ts-2-thread-1] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - HikariPool-1 - Connection is not available, request timed out after 634223ms.
2021-11-12 00:23:48,058 [sql-queue-0-ts-2-thread-1] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - This connection has been closed.
2021-11-12 00:23:48,022 [sql-queue-0-ts latest-6-thread-1] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - HikariPool-1 - Connection is not available, request timed out after 634223ms.
2021-11-12 00:23:48,059 [sql-queue-0-ts latest-6-thread-1] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - This connection has been closed.
2021-11-12 00:23:48,022 [sql-queue-2-ts latest-8-thread-1] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - HikariPool-1 - Connection is not available, request timed out after 624177ms.
2021-11-12 00:23:48,059 [sql-queue-2-ts latest-8-thread-1] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - This connection has been closed.
2021-11-12 00:23:48,023 [sql-queue-2-ts-4-thread-1] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - HikariPool-1 - Connection is not available, request timed out after 627819ms.
2021-11-12 00:23:48,059 [sql-queue-2-ts-4-thread-1] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - This connection has been closed.最后：
2021-11-12 00:33:10,919 [sql-queue-0-ts latest-6-thread-1] ERROR o.t.s.dao.sql.TbSqlBlockingQueue - [TS Latest] Failed to save 1 entities
org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:448)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.startTransaction(AbstractPlatformTransactionManager.java:400)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373)
at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:574)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:361)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692)
at org.thingsboard.server.dao.sqlts.insert.latest.psql.PsqlLatestInsertTsRepository$$EnhancerBySpringCGLIB$$381b448c.saveOrUpdate(<generated>)
at org.thingsboard.server.dao.sqlts.SqlTimeseriesLatestDao.lambda$init$3(SqlTimeseriesLatestDao.java:133)
at org.thingsboard.server.dao.sql.TbSqlBlockingQueue.lambda$init$2(TbSqlBlockingQueue.java:71)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection
at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:48)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:111)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:138)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getConnectionForTransactionManagement(LogicalConnectionManagedImpl.java:276)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.begin(LogicalConnectionManagedImpl.java:284)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.begin(JdbcResourceLocalTransactionCoordinatorImpl.java:246)
at org.hibernate.engine.transaction.internal.TransactionImpl.begin(TransactionImpl.java:83)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.beginTransaction(HibernateJpaDialect.java:184)
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:402)
... 16 common frames omitted
Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 634223ms.
at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:695)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:197)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:162)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128)
at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122)
at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:38)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:108)
... 23 common frames omitted
Caused by: org.postgresql.util.PSQLException: This connection has been closed.
at org.postgresql.jdbc.PgConnection.checkClosed(PgConnection.java:877)
at org.postgresql.jdbc.PgConnection.setNetworkTimeout(PgConnection.java:1610)
at com.zaxxer.hikari.pool.PoolBase.setNetworkTimeout(PoolBase.java:560)
at com.zaxxer.hikari.pool.PoolBase.isConnectionAlive(PoolBase.java:173)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186)
... 28 common frames omitted有趣的是，CPU负载上的时间戳与日志中的错误消息没有精确的关联。我为长期的错误信息道歉，但现在我不知道什么可能是根本原因。
我还没有试着重新安装整个电脑。
我的问题是，我该怎么做？有没有人曾遇到过类似的问题？我应该检查哪些日志/服务/等等，以掌握根本原因？
我应该尝试使用一台资源更多的机器吗？我应该尝试其他数据库和队列服务吗？
在当前的表单中，这个Thingsboard实例即使在测试中也不稳定。
编辑:对不起，我不能正确格式化错误代码的第一部分。Edit2:第一个链接是错误的。
回答 2
Stack Overflow用户
发布于 2021-12-12 21:26:19
在我将内存从1GB增加到4GB(从1GB)之后，ThingsBoard服务器就没有问题了。没有零星的冰冻。由于对这个问题没有其他可证明的建议，而且现在我的系统没有问题，所以我认为这个问题得到了回答。
Stack Overflow用户
发布于 2021-12-20 07:53:33
https://stackoverflow.com/questions/69996729
复制相似问题
腾讯云开发者
