blocks|key|5331307|text|如下所示：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|5331308|char+s[256];
strcpy(s,+"one+two+three");
char*+token+=+strtok(s,+"+");
while+(token)+{
++++printf("token:+%25s\n",+token);
++++token+=+strtok(NULL,+"+");
}|code-block|syntax|javascript|5331309|注意：strtok修改了字符串的标记化，所以它不能是const+char*。|offset|length|style|CODE|5331310|entityMap^0|0|0|3|6|Q|B|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|P|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|Q|8|@$I|R|J|S|K|L]|$I|T|J|U|K|L]]|9|@]|A|$]]|$1|M|3|-4|5|6|7|V|8|@]|9|@]|A|$]]]|N|$]]

Do it like this:

<pre><code>char s[256];
strcpy(s, "one two three");
char* token = strtok(s, " ");
while (token) {
 printf("token: %s\n", token);
 token = strtok(NULL, " ");
}
</code></pre>

Note: <code>strtok</code> modifies the string its tokenising, so it cannot be a <code>const char*</code>.

blocks|key|1803088|text|以下是strtok用法的一个示例，请记住，strtok对其输入字符串是破坏性的(因此不能用于字符串常量|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1803089|char+*p+=+strtok(str,+"+");
while(p+!=+NULL)+{
++++printf("%25s\n",+p);
++++p+=+strtok(NULL,+"+");
}|code-block|syntax|javascript|1803090|基本上需要注意的是，将NULL作为第一个参数传递给strtok会告诉它从之前标记的字符串中获取下一个标记。|1803091|entityMap^0|3|6|L|6|0|0|B|4|P|6|0^^$0|@$1|2|3|4|5|6|7|O|8|@$9|P|A|Q|B|C]|$9|R|A|S|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|T|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|U|8|@$9|V|A|W|B|C]|$9|X|A|Y|B|C]]|D|@]|E|$]]|$1|M|3|-4|5|6|7|Z|8|@]|D|@]|E|$]]]|N|$]]

Here's an example of <code>strtok</code> usage, keep in mind that <code>strtok</code> is destructive of its input string (and therefore can't ever be used on a string constant

<pre><code>char *p = strtok(str, " ");
while(p != NULL) {
 printf("%s\n", p);
 p = strtok(NULL, " ");
}
</code></pre>

Basically the thing to note is that passing a <code>NULL</code> as the first parameter to <code>strtok</code> tells it to get the next token from the string it was previously tokenizing.

blocks|key|3913620|text|strtok可能是非常危险的。它不是线程安全的。它的预期用途是在循环中一遍又一遍地调用，传递上一个调用的输出。strtok函数有一个存储strtok调用状态的内部变量。这种状态对于每个线程来说并不是唯一的-它是全局的。如果任何其他代码在另一个线程中使用strtok，您就会遇到问题。也不是你想要追踪的那种问题！|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3913621|我建议使用regex实现，或者使用sscanf来拆分字符串。|3913622|试试这个：|3913623|char+strprint[256];
char+text[256];
strcpy(text,+"My+string+to+test");
while+(+sscanf(+text,+"%25s+%25s",+strprint,+text)+>+0+)+{
+++printf("token:+%25s\n",+strprint);
}|code-block|syntax|javascript|3913624|注意：“text”字符串在被分隔时会被销毁。这可能不是首选行为=)|3913625|entityMap^0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|P|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|Q|8|@]|9|@]|A|$]]|$1|F|3|G|5|H|7|R|8|@]|9|@]|A|$I|J]]|$1|K|3|L|5|6|7|S|8|@]|9|@]|A|$]]|$1|M|3|-4|5|6|7|T|8|@]|9|@]|A|$]]]|N|$]]

strtok can be very dangerous. It is not thread safe. Its intended use is to be called over and over in a loop, passing in the output from the previous call. The strtok function has an internal variable that stores the state of the strtok call. This state is not unique to each thread - it is global. If any other code uses strtok in another thread, you get problems. Not the kind of problems you want to track down either!

I'd recommend looking for a regex implementation, or using sscanf to pull apart the string.

Try this:

<pre><code>char strprint[256];
char text[256];
strcpy(text, "My string to test");
while ( sscanf( text, "%s %s", strprint, text) &gt; 0 ) {
 printf("token: %s\n", strprint);
}
</code></pre>

Note: The 'text' string is destroyed as it's separated. This may not be the preferred behaviour =)

blocks|key|3913608|text|您可以通过引入一个额外的变量来简化代码。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3913609|#include+<string.h>
#include+<stdio.h>

int+main()
{
++++char+str[100],+*s+=+str,+*t+=+NULL;

++++strcpy(str,+"a+space+delimited+string");
++++while+((t+=+strtok(s,+"+"))+!=+NULL)+{
++++++++s+=+NULL;
++++++++printf(":%25s:\n",+t);
++++}
++++return+0;
}|code-block|syntax|javascript|3913610|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

You can simplify the code by introducing an extra variable.

<pre><code>#include &lt;string.h&gt;
#include &lt;stdio.h&gt;

int main()
{
 char str[100], *s = str, *t = NULL;

 strcpy(str, "a space delimited string");
 while ((t = strtok(s, " ")) != NULL) {
 s = NULL;
 printf(":%s:\n", t);
 }
 return 0;
}
</code></pre>

blocks|key|3913658|text|我制作了一些字符串函数来拆分值，尽可能少地使用指针，因为这段代码打算在PIC18F处理器上运行。当你只有几个可用的RAM时，这些处理器并不能很好地处理指针：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3913659|#include+<stdio.h>
#include+<string.h>

char+POSTREQ[255]+=+"pwd=123456&apply=Apply&d1=88&d2=100&pwr=1&mpx=Internal&stmo=Stereo&proc=Processor&cmp=Compressor&ip1=192&ip2=168&ip3=10&ip4=131&gw1=192&gw2=168&gw3=10&gw4=192&pt=80&lic=&A=A";

int+findchar(char+*string,+int+Start,+char+C)+{
++++while((string[Start]+!=+0))+{+Start%2B%2B;+if(string[Start]+==+C)+return+Start;+}
++++return+-1;
}

int+findcharn(char+*string,+int+Times,+char+C)+{
+++int+i+=+0,+pos+=+0,+fnd+=+0;

++++while(i+<+Times)+{
+++++++fnd+=+findchar(string,+pos,+C);
++++++++if(fnd+<+0)+return+-1;
++++++++if(fnd+>+0)+pos+=+fnd;
+++++++i%2B%2B;
+++}
+++return+fnd;
}

void+mid(char+*in,+char+*out,+int+start,+int+end)+{
++++int+i+=+0;
++++int+size+=+end+-+start;

++++for(i+=+0;+i+<+size;+i%2B%2B){
++++++++out[i]+=+in[start+%2B+i+%2B+1];
++++}
++++out[size]+=+0;
}

void+getvalue(char+*out,+int+index)+{
++++mid(POSTREQ,+out,+findcharn(POSTREQ,+index,+'='),+(findcharn(POSTREQ,+index,+'&')+-+1));
}

void+main()+{
+++char+n_pwd[7];
+++char+n_d1[7];

+++getvalue(n_d1,+1);

+++printf("Value:+%25s\n",+n_d1);
}+|code-block|syntax|javascript|3913660|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

I've made some string functions in order to split values, by using less pointers as I could because this code is intended to run on PIC18F processors. Those processors does not handle really good with pointers when you have few free RAM available:

<pre><code>#include &lt;stdio.h&gt;
#include &lt;string.h&gt;

char POSTREQ[255] = "pwd=123456&amp;apply=Apply&amp;d1=88&amp;d2=100&amp;pwr=1&amp;mpx=Internal&amp;stmo=Stereo&amp;proc=Processor&amp;cmp=Compressor&amp;ip1=192&amp;ip2=168&amp;ip3=10&amp;ip4=131&amp;gw1=192&amp;gw2=168&amp;gw3=10&amp;gw4=192&amp;pt=80&amp;lic=&amp;A=A";

int findchar(char *string, int Start, char C) {
 while((string[Start] != 0)) { Start++; if(string[Start] == C) return Start; }
 return -1;
}

int findcharn(char *string, int Times, char C) {
 int i = 0, pos = 0, fnd = 0;

 while(i &lt; Times) {
 fnd = findchar(string, pos, C);
 if(fnd &lt; 0) return -1;
 if(fnd &gt; 0) pos = fnd;
 i++;
 }
 return fnd;
}

void mid(char *in, char *out, int start, int end) {
 int i = 0;
 int size = end - start;

 for(i = 0; i &lt; size; i++){
 out[i] = in[start + i + 1];
 }
 out[size] = 0;
}

void getvalue(char *out, int index) {
 mid(POSTREQ, out, findcharn(POSTREQ, index, '='), (findcharn(POSTREQ, index, '&amp;') - 1));
}

void main() {
 char n_pwd[7];
 char n_d1[7];

 getvalue(n_d1, 1);

 printf("Value: %s\n", n_d1);
} 
</code></pre>

blocks|key|1803075|text|在阅读strtok文档时，我发现您需要在第一次“初始化”调用之后传入一个空指针。也许你没那么做。当然，这只是一个猜测。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1803076|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

When reading the strtok documentation, I see you need to pass in a NULL pointer after the first "initializing" call. Maybe you didn't do that. Just a guess of course.

blocks|key|5331511|text|下面是另一个strtok()实现，它能够识别连续的分隔符(标准库的strtok()没有这个功能)|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|5331512|该函数是BSD许可字符串库的一部分，称为zString。非常欢迎您的贡献:)|5331513|https://github.com/fnoyanisi/zString|5331514|char+*zstring_strtok(char+*str,+const+char+*delim)+{
++++static+char+*static_str=0;++++++/*+var+to+store+last+address+*/
++++int+index=0,+strlength=0;+++++++/*+integers+for+indexes+*/
++++int+found+=+0;++++++++++++++++++/*+check+if+delim+is+found+*/

++++/*+delimiter+cannot+be+NULL
++++*+if+no+more+char+left,+return+NULL+as+well
++++*/
++++if+(delim==0+%7C%7C+(str+==+0+&&+static_str+==+0))
++++++++return+0;

++++if+(str+==+0)
++++++++str+=+static_str;

++++/*+get+length+of+string+*/
++++while(str[strlength])
++++++++strlength%2B%2B;

++++/*+find+the+first+occurance+of+delim+*/
++++for+(index=0;index<strlength;index%2B%2B)
++++++++if+(str[index]==delim[0])+{
++++++++++++found=1;
++++++++++++break;
++++++++}

++++/*+if+delim+is+not+contained+in+str,+return+str+*/
++++if+(!found)+{
++++++++static_str+=+0;
++++++++return+str;
++++}

++++/*+check+for+consecutive+delimiters
++++*if+first+char+is+delim,+return+delim
++++*/
++++if+(str[0]==delim[0])+{
++++++++static_str+=+(str+%2B+1);
++++++++return+(char+*)delim;
++++}

++++/*+terminate+the+string
++++*+this+assignmetn+requires+char[],+so+str+has+to
++++*+be+char[]+rather+than+*char
++++*/
++++str[index]+=+'\0';

++++/*+save+the+rest+of+the+string+*/
++++if+((str+%2B+index+%2B+1)!=0)
++++++++static_str+=+(str+%2B+index+%2B+1);
++++else
++++++++static_str+=+0;

++++++++return+str;
}|code-block|syntax|javascript|5331515|正如在之前的文章中提到的，由于strtok()或我在上面实现的应用程序依赖于static+*char变量来保留连续调用之间的最后一个分隔符的位置，所以在处理多线程应用程序时应该格外小心。|5331516|entityMap|0|LINK|mutability|MUTABLE|url|1^0|6|8|X|8|0|K|7|0|0|0|10|1|0|0|F|8|12|C|0^^$0|@$1|2|3|4|5|6|7|Y|8|@$9|Z|A|10|B|C]|$9|11|A|12|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|13|8|@]|D|@$9|14|A|15|1|16]]|E|$]]|$1|H|3|I|5|6|7|17|8|@]|D|@$9|18|A|19|1|1A]]|E|$]]|$1|J|3|K|5|L|7|1B|8|@]|D|@]|E|$M|N]]|$1|O|3|P|5|6|7|1C|8|@$9|1D|A|1E|B|C]|$9|1F|A|1G|B|C]]|D|@]|E|$]]|$1|Q|3|-4|5|6|7|1H|8|@]|D|@]|E|$]]]|R|$S|$5|T|U|V|E|$W|I]]|X|$5|T|U|V|E|$W|I]]]]

Here is another <code>strtok()</code> implementation, which has the ability to recognize consecutive delimiters (standard library's <code>strtok()</code> does not have this)

The function is a part of BSD licensed string library, called <a href="https://github.com/fnoyanisi/zString" rel="nofollow">zString</a>. You are more than welcome to contribute :)

<a href="https://github.com/fnoyanisi/zString" rel="nofollow">https://github.com/fnoyanisi/zString</a>

<pre><code>char *zstring_strtok(char *str, const char *delim) {
 static char *static_str=0; /* var to store last address */
 int index=0, strlength=0; /* integers for indexes */
 int found = 0; /* check if delim is found */

 /* delimiter cannot be NULL
 * if no more char left, return NULL as well
 */
 if (delim==0 || (str == 0 &amp;&amp; static_str == 0))
 return 0;

 if (str == 0)
 str = static_str;

 /* get length of string */
 while(str[strlength])
 strlength++;

 /* find the first occurance of delim */
 for (index=0;index&lt;strlength;index++)
 if (str[index]==delim[0]) {
 found=1;
 break;
 }

 /* if delim is not contained in str, return str */
 if (!found) {
 static_str = 0;
 return str;
 }

 /* check for consecutive delimiters
 *if first char is delim, return delim
 */
 if (str[0]==delim[0]) {
 static_str = (str + 1);
 return (char *)delim;
 }

 /* terminate the string
 * this assignmetn requires char[], so str has to
 * be char[] rather than *char
 */
 str[index] = '\0';

 /* save the rest of the string */
 if ((str + index + 1)!=0)
 static_str = (str + index + 1);
 else
 static_str = 0;

 return str;
}
</code></pre>

As mentioned in previous posts, since <code>strtok()</code>, or the one I implmented above, relies on a <code>static *char</code> variable to preserve the location of last delimiter between consecutive calls, extra care should be taken while dealing with multi-threaded aplications.

blocks|key|1803250|text|int+not_in_delimiter(char+c,+char+*delim){

++++while(*delim+!=+'\0'){
++++++++++++if(c+==+*delim)+return+0;
++++++++++++delim%2B%2B;
++++}
++++return+1;
}

char+*token_separater(char+*source,+char+*delimiter,+char+**last){

char+*begin,+*next_token;
char+*sbegin;

/*Get+the+start+of+the+token+*/
if(source)
++begin+=+source;
else
++begin+=+*last;

sbegin+=+begin;

/*Scan+through+the+string+till+we+find+character+in+delimiter.+*/
while(*begin+!=+'\0'+&&+not_in_delimiter(*begin,+delimiter)){
+++++++begin%2B%2B;
}

/*+Check+if+we+have+reached+at+of+the+string+*/
if(*begin+==+'\0')+{
/*+We+dont+need+to+come+further,+hence+return+NULL*/
+++*last+=+NULL;
++++return+sbegin;
}
/*+Scan+the+string+till+we+find+a+character+which+is+not+in+delimiter+*/
+next_token++=+begin;
+while(next_token+!=+'\0'+&&+!not_in_delimiter(*next_token,+delimiter))++++{
++++next_token%2B%2B;
+}
+/*+If+we+have+not+reached+at+the+end+of+the+string+*/
+if(*next_token+!=+'\0'){
++*last+=+next_token--;
++*next_token+=+'\0';
+++return+sbegin;
}
}

+void+main(){

++++char+string[10]+=+"abcb_dccc";
++++char+delim[10]+=+"_";
++++char+*token+=+NULL;
++++char+*last+=+""+;
++++token++=+token_separater(string,+delim,+&last);
++++printf("%25s\n",+token);
++++while(last){
++++++++++++token++=+token_separater(NULL,+delim,+&last);
++++++++++++printf("%25s\n",+token);
++++}|type|code-block|depth|inlineStyleRanges|entityRanges|data|syntax|javascript|1803251|}|unstyled|1803252|你可以在我的个人资料中提到的博客上阅读详细分析:)|1803253|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@]|9|@]|A|$B|C]]|$1|D|3|E|5|F|7|L|8|@]|9|@]|A|$]]|$1|G|3|H|5|F|7|M|8|@]|9|@]|A|$]]|$1|I|3|-4|5|F|7|N|8|@]|9|@]|A|$]]]|J|$]]

<pre><code>int not_in_delimiter(char c, char *delim){

 while(*delim != '\0'){
 if(c == *delim) return 0;
 delim++;
 }
 return 1;
}

char *token_separater(char *source, char *delimiter, char **last){

char *begin, *next_token;
char *sbegin;

/*Get the start of the token */
if(source)
 begin = source;
else
 begin = *last;

sbegin = begin;

/*Scan through the string till we find character in delimiter. */
while(*begin != '\0' &amp;&amp; not_in_delimiter(*begin, delimiter)){
 begin++;
}

/* Check if we have reached at of the string */
if(*begin == '\0') {
/* We dont need to come further, hence return NULL*/
 *last = NULL;
 return sbegin;
}
/* Scan the string till we find a character which is not in delimiter */
 next_token = begin;
 while(next_token != '\0' &amp;&amp; !not_in_delimiter(*next_token, delimiter)) {
 next_token++;
 }
 /* If we have not reached at the end of the string */
 if(*next_token != '\0'){
 *last = next_token--;
 *next_token = '\0';
 return sbegin;
}
}

 void main(){

 char string[10] = "abcb_dccc";
 char delim[10] = "_";
 char *token = NULL;
 char *last = "" ;
 token = token_separater(string, delim, &amp;last);
 printf("%s\n", token);
 while(last){
 token = token_separater(NULL, delim, &amp;last);
 printf("%s\n", token);
 }
</code></pre>

}

You can read detail analysis at blog mentioned in my profile :)

I have been trying to tokenize a string using SPACE as delimiter but it doesn't work. Does any one have suggestion on why it doesn't work?

Edit: tokenizing using:

<pre><code>strtok(string, " ");
</code></pre>

The code is like the following

<pre><code>pch = strtok (str," ");
while (pch != NULL)
{
 printf ("%s\n",pch);
 pch = strtok (NULL, " ");
}
</code></pre>

Tokenizing strings in C

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我一直在尝试使用空格作为分隔符来标记一个字符串，但它不起作用。有没有人对为什么它不起作用有什么建议？编辑:标记化使用：strtok(string, " ");代码如下所示pch = strtok (str," ");while (pch != NULL){  printf ("%s\n",pch);  pch = strtok (NULL, " ");}

问C++中字符串的标记化
EN

回答 8

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问C++中字符串的标记化EN

回答 8

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问C++中字符串的标记化
EN