ABAP开发的Github issue备份工具

Jerry Wang

发布于 2020-08-31 10:19:45

4420

发布于 2020-08-31 10:19:45

文章被收录于专栏：Jerry的SAP技术分享 Jerry的SAP技术分享Jerry的SAP技术分享

1. Basic idea of this tiny issue backup tool implemented by ABAP
1.1 Issues should be persisted to ABAP server
1.2 Delta-backup functionality support
1.3 Paging in issue API needs to be considered
1.4 API Rate Limiting is not considered for simplification reason
2. How to use this tool
3. More features to be added
3.1 issue label information
3.2 CDS views for analysis purpose
3.3 image backup

I personally prefer to use Github repository issue to manage my knowledge. For issue, this is my github repository to store ABAP and SAP CRM related knowledge of mine and currently it has already 285 issues.

There are two main reasons which drive me to choose github repository as my knowledge management approach:

great speed of full text search
powerful and flexible label management of issues

As the accumulation of issue number, I have to consider the possibility of issue backup, so that I can still have access to my knowledge repository when the public github is not available ( for example due to network issue or access within China banned by government due to political reasons, who knows ). There are plenty of articles and tools about how to export/backup github repository issues:

However after I go through them, I found none of the tool can 100% fulfill my backup requirement, as a result I plan to write one backup too on my own using github public API documented here.

Basic idea of this tiny issue backup tool implemented by ABAP

1.1 Issues should be persisted to ABAP server

Since as an ABAPer we have powerful Netweaver behind us, I plan to develop a tool which can extract all issues of a given repository and stored them in an ABAP system, so that I can use all kinds of ABAP tools for example CDS view to make analysis on those issues later.

Only a single API endpoint is used in this tool, see how to use it below.

The format of this endpoint url is: https://api.github.com/repos///issues

In my example above, the response is a JSON string which is an array with each element representing an issue detail.

For me I am only interested with issue number, issue title, issue body text, created and update timestamp, so I create the following database table to store github issue information:

1.2 Delta-backup functionality support

In github repository each issue is marked with a number, which could be easily found in issue url and issue detail page.

The delta backup function means for example I have finished the backup of current 285 issues, and later when I create two new issues 286 and 287, and then I run the tool again, only 286 and 287 should now be exported and persisted into ABAP system.

1.3 Paging in issue API needs to be considered

In current Github REST API V3, by default only 30 issues are returned in a single API call.

If there are still left issues to be exported, there will be two indicators stored in HTTP response fields of this single API call, one (rel=”next”) contains url which returns the next 30 issues and the other(rel=”last”) contains the url which returns the issues in the last page ). This paging logic should also be supported in the tool.

1.4 API Rate Limiting is not considered for simplification reason

For more detail about API rate limiting topic, refer to Github API documentation here.

How to use this tool

(1) Create one transparent table to store issue detail

And fill one entry for the github repository whose issue you would like to extract accordingly.

You can choose any name in column REPO_NAME which acts as an internal key for the github repository name.

(2) execute method START_BACKUP of tool class, specify the internal repository name you specified in the first step.

Once finished, you could find that all issues are stored into table CRMD_GIT_ISSUE.

(3) The body of a given issue could be downloaded locally using method DOWNLOAD_ISSUE, just specify the repository name, issue number and the path of local folder where the github issue file you would like to store:

Since most of my github issue contains Chinese characters, so I use code page 8400 in the code.

here below is the original issue opened in the browser:

Here below is the downloaded issue source code in markdown format:

More features to be added

I will enhance the tool with following features in the future

3.1 issue label information

Multiple labels could be maintained for a single issue and it is very convenient to search by these maintained labels.

Currently the label information is not stored in the database table.

3.2 CDS views for analysis purpose

Since now we have lots of content in our database table, we could develop several CDS view to meet with our specific requirement, for example:

calculate the total length of all issues belonging to a given github repository
get an overview of all labels and the concrete number of issues assigned with each label

Updated on 2017-07-17

Get an overview of how many issues created for each repository

Solution could be found from CDS view CRMV_GIT_ISSUE.

Get how many issues created per day

Solution could be found from CDS view CRMV_ISSUE_CREATION_DATE_COUNT.

Get an overview of how many images used in a given github issue

Solution could be found from CDS view CRMV_GIT_ISSUE_IMAGE_NUM.

3.3 image backup

Currently only the issue source code with markdown format is extracted and stored in database table in ABAP server. However most of my issues contains screenshot and in the issue source code only a reference to these image files are there – the binary content of image files themselves are not stored in ABAP server. This could be enhanced in the future.