net_pkt_append() has been changed. If payload reached max value
of 'data_len' in net packet, net_pkt_append will not append.
So the caller has to create new packet and append remaining payload.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@linux.intel.com>
Logic for sending chunks of data is incompatible with adding
Content-Length: header.
Per https://tools.ietf.org/html/rfc7230#section-3.3.1:
"A sender MUST NOT send a Content-Length header field in any
message that contains a Transfer-Encoding header field."
Going a bit further in my mind: also don't send Transfer-Encoded
chunked data either when the Content-Length header is present.
In general, there will be problems if the http client library
makes payload changes without the user code knowing about it.
This patch removes the use of http_send_chunk() from the new
HTTP client code and instead sends the payload directly to
http_prepare_and_send()
This fixes an issue where every available buffer would be allocated
with repeating payload data because the for loop in http_request()
wasn't ending until we ran out of memory.
Signed-off-by: Michael Scott <michael@opensourcefoundries.com>
We should not use the user suppied timeout setting in
http_client_send_req() for the connection timeout. In the
previous API the call to tcp_connect() used
CONFIG_HTTP_CLIENT_NETWORK_TIMEOUT as the timeout setting.
Let's do that here too.
This fixes -ETIMEDOUT error generation when using K_NO_WAIT
for http_client_send_req().
Signed-off-by: Michael Scott <michael@opensourcefoundries.com>
Add status error string when sending a error message from
HTTP server to client as described in RFC 2616 ch 6.1.
Previously only error code was sent except for 400 (Bad Request).
This also fixes uninitialized memory access in error message.
Coverity-CID: 178792
Fixes#4782
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Introducing CMake is an important step in a larger effort to make
Zephyr easy to use for application developers working on different
platforms with different development environment needs.
Simplified, this change retains Kconfig as-is, and replaces all
Makefiles with CMakeLists.txt. The DSL-like Make language that KBuild
offers is replaced by a set of CMake extentions. These extentions have
either provided simple one-to-one translations of KBuild features or
introduced new concepts that replace KBuild concepts.
This is a breaking change for existing test infrastructure and build
scripts that are maintained out-of-tree. But for FW itself, no porting
should be necessary.
For users that just want to continue their work with minimal
disruption the following should suffice:
Install CMake 3.8.2+
Port any out-of-tree Makefiles to CMake.
Learn the absolute minimum about the new command line interface:
$ cd samples/hello_world
$ mkdir build && cd build
$ cmake -DBOARD=nrf52_pca10040 ..
$ cd build
$ make
PR: zephyrproject-rtos#4692
docs: http://docs.zephyrproject.org/getting_started/getting_started.html
Signed-off-by: Sebastian Boe <sebastian.boe@nordicsemi.no>
Create http library that uses net-app instead of net_context
directly. The old HTTP API is deprecated.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Recent commit fb7f6cfa97 ("net: lib: http: Fix invalid pointer
body_start") introduced logic to reset the response body_start pointer
when the response buffer was reused.
This check needs to be fixed so that it doesn't arbitrarily change
body_start when not needed.
The problem with the current check can be demonstrated by not setting
a response callback for request which generates a large response
spanning multiple packets.
In this case body_start is still valid (not reusing the response buffer
because there is no callback set), but it will be changed when the 2nd
packet is received and the "at" marker is located at the head of the
new packet (!= response_buffer).
Signed-off-by: Michael Scott <michael.scott@linaro.org>
Inform user if we could not install receive callback after
a connection is accepted in http server.
Coverity-CID: 178244
Fixes#4584
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
The body_start field at http_client_ctx.rsp is used to check if this
fragment contains (a part of) headers or not.
If the device recived more than one fragment in one http response,
may cause re-use of the result buffer in function on_body().
Once the device re-use the result buffer, the body_start that point
to this buffer address will no longer be valid.
Signed-off-by: Ding Tao <miyatsu@qq.com>
Currently this is defined as a k_thread_stack_t pointer.
However this isn't correct, stacks are defined as arrays. Extern
references to k_thread_stack_t doesn't work properly as the compiler
treats it as a pointer to the stack array and not the array itself.
Declaring as an unsized array of k_thread_stack_t doesn't work
well either. The least amount of confusion is to leave out the
pointer/array status completely, use pointers for function prototypes,
and define K_THREAD_STACK_EXTERN() to properly create an extern
reference.
The definitions for all functions and struct that use
k_thread_stack_t need to be updated, but code that uses them should
be unchanged.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
With the introduction of CoAP and other protocols, URL parsing is
be needed when HTTP_PARSER is not. Let's split out the existing
functionality of URL parsing into it's own CONFIG and let
HTTP_PARSER use it by automatically selecting HTTP_PARSER_URL when
HTTP_PARSER is enabled.
Signed-off-by: Michael Scott <michael.scott@linaro.org>
If the caller of http_client_send_req() sets the timeout to
K_NO_WAIT, then the function would still wait for a while before
returning to the caller.
Jira: ZEP-2624
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
k_delayed_work_cancel now only fail if it hasn't been submitted which
means it is not in use anyway so it safe to reset its data regardless
of its return.
Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
The original commit 8ebaf29927 ("net: http: dont timeout
on HTTP requests w/o body") was intended to handle a case
where an HTTP response had been retrieved from the server but
the HTTP parser couldn't meet the criteria for calling
"on_message_complete". For example, a POST to a REST API
where the server doesn't return anything but an HTTP
status code.
It was a really bad idea to check a semaphore count. There
is a lot of kernel logic built into semaphores and how the
count is adjusted. The assumption that the value is 0
after the k_sem_give() is incorrect. It's STILL 0 if
something is pending with a k_sem_take(). By the time
k_sem_give() is done executing the other thread has now
been kicked and the count is back to 0.
This caused the original check to always pass and in turn
breakage was noticed in the http_client sample.
Let's do this the right way by setting a flag when
on_message_complete is called and if that flag is not set
by the time we reach recv_cb, let's give back the semaphore
to avoid a timeout.
Jira: ZEP-2561
Signed-off-by: Michael Scott <michael.scott@linaro.org>
The HTTP header field pointers are saved for each HTTP request.
But the counter that saves the pointers was never reset to initial
value when the connection was dropped. This meant that the header
field values were only proper for first HTTP request.
Jira: ZEP-2463
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
POSIX requires struct sockaddr's field to be named "sa_family"
(not just "family"):
http://pubs.opengroup.org/onlinepubs/009696699/basedefs/sys/socket.h.html
This change allows to port POSIX apps easier (including writing
portable apps using BSD Sockets compatible API).
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Make sure that a network packet is sent after calling
http_response_send_data(). Othwerwise the packets might be
piling up and not sent in timely manner.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Currently, the HTTP_NETWORK_TIMEOUT setting is hard-coded as 20 seconds.
Not every application may want to wait that long, so let's change this
to a CONFIG option: CONFIG_HTTP_CLIENT_NETWORK_TIMEOUT
NOTE: This also removes HTTP_NETWORK_TIMEOUT from the public http.h
include file. It was not being used externally to HTTP client sources.
Signed-off-by: Michael Scott <michael.scott@linaro.org>
A TCP FIN message is passed on to user apps as a tcp_received_callback
with a NULL pkt parameter. This means the connection is closing and
the app should do whatever cleanup it needs as there will be no further
callbacks for the current TCP connection.
Currently, if a HTTP client request doesn't receive a "body" which
the HTTP parser can use to trigger on_message_complete, then the request
will end up timing out and most apps will think an error has occurred.
Instead, let's handle the TCP FIN message and return the waiting
semaphore, leaving the app to deal with whatever has been set in the
current HTTP context response data (IE: http_status).
This fixes using HTTP client to send POST data to servers which
only respond with HTTP_OK status and no body.
Signed-off-by: Michael Scott <michael.scott@linaro.org>
RFC-7230 "HTTP/1.1 Message Syntax and Routing" Section 5.4
describes the "Host" header formatting. If Zephyr user
specifies a host string as a part of the HTTP client request
structure, we end up sending an incorrect HTTP header due
to a missing "Host :" text.
Fix this by prepending "Host: " to the header data before
the user supplied host string.
Signed-off-by: Michael Scott <michael.scott@linaro.org>
Historically, stacks were just character buffers and could be treated
as such if the user wanted to look inside the stack data, and also
declared as an array of the desired stack size.
This is no longer the case. Certain architectures will create a memory
region much larger to account for MPU/MMU guard pages. Unfortunately,
the kernel interfaces treat both the declared stack, and the valid
stack buffer within it as the same char * data type, even though these
absolutely cannot be used interchangeably.
We introduce an opaque k_thread_stack_t which gets instantiated by
K_THREAD_STACK_DECLARE(), this is no longer treated by the compiler
as a character pointer, even though it really is.
To access the real stack buffer within, the result of
K_THREAD_STACK_BUFFER() can be used, which will return a char * type.
This should catch a bunch of programming mistakes at build time:
- Declaring a character array outside of K_THREAD_STACK_DECLARE() and
passing it to K_THREAD_CREATE
- Directly examining the stack created by K_THREAD_STACK_DECLARE()
which is not actually the memory desired and may trigger a CPU
exception
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This commit adds http_client_set_net_pkt_pool() function that allows
caller to define net_buf pool that is used when sending a TCP packet.
This is needed for those technologies like Bluetooth or 802.15.4 which
compress the IPv6 header during send.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
This commit adds http_server_set_net_pkt_pool() function that allows
caller to define net_buf pool that is used when sending a TCP packet.
This is needed for those technologies like Bluetooth or 802.15.4 which
compress the IPv6 header during send.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Use net_context_put(), not net_context_unref(). This makes sure
that after sending response, connections are properly closed.
Jira: ZEP-2362
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Coverity reported false positives, add comment about these in
the code.
Jira: ZEP-2344
Jira: ZEP-2345
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
The global mbedtls heap is set automatically now so no need to
set it individually in the http library.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
If we receive a HTTP request and if the earlier context is still
active and it is not the same as the new one, then close the earlier
one. Otherwise it is possible that the old context will be left into
TCP ESTABLISHED state and would never be released. Example of this
is that we had IPv4 connection active and then IPv6 connection is
established, in this case we will disconnect the IPv4 connection
after this commit.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
The "net http monitor" command turns on HTTP monitoring,
which means that for each incoming HTTP or HTTPS request,
a information about source and destination address, and
the HTTP request URL is printed.
User can disable the monitoring by "net http" command.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
If CONFIG_NET_DEBUG_HTTP_CONN is enabled, then start to collect
currently active HTTP connections to HTTP server.
This is only useful for debugging the HTTP connections.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Add HTTPS support into http-client library. The init of the
HTTPS client connection is different compared to HTTP client,
but the actual HTTP request sending is using the same API as
HTTP client.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
This is done so that both http_client and http_server functionality
can share the same heap.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
The mbedtls debugging function was set before the ssl config
struct was initialized. This meant that it was not possible
to activate mbedtls debug prints. This commit sets the debug
print option after the config struct has been initialized.
Fixed also the debug prints which print extra \n which looks
very bad in debugging outputs.
This commit does not enable mbedtls debugging, it just makes it
possible to output mbedtls debug prints. In order to get mbedlts
debug prints one needs to do this:
* set DEBUG_THRESHOLD to >0 in http_server.c
* enable CONFIG_NET_DEBUG_HTTP in project config file
* enable MBEDTLS_DEBUG_C in mbedtls config file (see file pointed
by CONFIG_MBEDTLS_CFG_FILE option)
* in qemu, one needs to increase the size of the available RAM,
this setting does the trick, CONFIG_RAM_SIZE=300
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
In some cases the net_pkt can be null when freeing it,
this will print error from net_pkt library. Avoid this by
checking the value of net_pkt before calling net_pkt_unref().
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
If we re-connect to same peer server, then we should select a new
source port. Noticed that if the same source port as before is
used for the new connection, the peer might drop the packet. This
was seen when connecting to Linux peer.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
If we received a bad HTTP request, then subsequent good requests
were also returning 400 error code. The parsing state needs to
be initialized after each received HTTP request.
Jira: ZEP-2181
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
If the HTTPS connection is closed, then properly handle call to
HTTP parser init in case of error and also remove any pending
data that belong to old connection.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Allow the caller to delay the closing of the HTTP connection
for a number of milliseconds. The purpose for this is that
the client can send still some data back to us for a short
period of time.
This is needed for example for Basic authentication so that
server is able to receive authentication values back.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
This commit creates a HTTP server library. So instead of creating
a complex HTTP server application for serving HTTP requests, the
developer can use the HTTP server API to create HTTP server
insteances. This commit also adds support for creating HTTPS servers.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Instead of separate sample application that does everything
related to HTTP client connectivity, create a HTTP client library
that hides nasty details that are related to sending HTTP methods.
After this the sample HTTP client application is very simple and
only shows how to use the client HTTP API.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
For stream-based protocols (TCP), adding less data than requested
("short write") is generally not a problem - the rest of data can
be sent in the next packet. So, make net_pkt_append() return length
of written data instead of just bool flag, which makes it closer
to the behavior of POSIX send()/write() calls.
There're many users of older net_pkt_append() in the codebase
however, so net_pkt_append_all() convenience function is added which
keeps returning a boolean flag. All current users were converted to
this function, except for two:
samples/net/http_server/src/ssl_utils.c
samples/net/mbedtls_sslclient/src/tcp.c
Both are related to TLS and implement mbedTLS "tx callback", which
follows POSIX short-write semantics. Both cases also had a code to
workaround previous boolean-only behavior of net_pkt_append() - after
calling it, they measured length of the actual data added (but only
in case of successful return of net_pkt_append(), so that didn't
really help). So, these 2 cases are already improved.
Jira: ZEP-1984
Change-Id: Ibaf7c029b15e91b516d73dab3612eed190ee982b
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Convert code to use u{8,16,32,64}_t and s{8,16,32,64}_t instead of C99
integer types.
Jira: ZEP-2051
Change-Id: I4ec03eb2183d59ef86ea2c20d956e5d272656837
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
- net_pkt becomes a stand-alone structure with network packet meta
information.
- network packet data is still managed through net_buf, mostly named
'frag'.
- net_pkt memory management is done through k_mem_slab
- function got introduced or relevantly renamed to target eithe net_pkt
or net_buf fragments.
- net_buf's sent_list ends up in net_pkt now, and thus helps to save
memory when TCP is enabled.
Change-Id: Ibd5c17df4f75891dec79db723a4c9fc704eb843d
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
There have been long lasting confusion between net_buf and net_nbuf.
While the first is actually a buffer, the second one is not. It's a
network buffer descriptor. More precisely it provides meta data about a
network packet, and holds the chain of buffer fragments made of net_buf.
Thus renaming net_nbuf to net_pkt and all names around it as well
(function, Kconfig option, ..).
Though net_pkt if the new name, it still inherit its logic from net_buf.
'
This patch is the first of a serie that will separate completely net_pkt
from net_buf.
Change-Id: Iecb32d2a0d8f4647692e5328e54b5c35454194cd
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Due to commit 8308b9bd2d ("net/http: Add the HTTP/1.1 API")
every user of CONFIG_HTTP_PARSER would need to add CONFIG_HTTP to
their .conf files. Which is fine for intree samples/tests as they
have been adjusted, but the rest of world working on Zephyr apps
will need to make this changes as well.
Instead, we should have each of the following select HTTP instead of
depend on it, which will make future use of these configs and their
dependencies more intuitive:
HTTP_SERVER
HTTP_CLIENT
HTTP_PARSER
NOTE: As cleanup, this commit also removes the CONFIG_HTTP added to
samples and test .conf files.
Change-Id: I81cfaa19e37333b1bf98778f8147814780e7f77c
Signed-off-by: Michael Scott <michael.scott@linaro.org>
Ravi kumar Veeramally