suyu/src/core/hle/service/ssl/ssl.cpp

553 lines
19 KiB
C++
Raw Normal View History

// SPDX-FileCopyrightText: Copyright 2018 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
2018-03-23 02:32:50 -04:00
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
#include "common/string_util.h"
#include "core/core.h"
2023-02-19 14:42:12 -05:00
#include "core/hle/service/ipc_helpers.h"
#include "core/hle/service/server_manager.h"
#include "core/hle/service/service.h"
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
#include "core/hle/service/sm/sm.h"
#include "core/hle/service/sockets/bsd.h"
2018-03-23 02:32:50 -04:00
#include "core/hle/service/ssl/ssl.h"
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
#include "core/hle/service/ssl/ssl_backend.h"
#include "core/internal_network/network.h"
#include "core/internal_network/sockets.h"
2018-03-23 02:32:50 -04:00
namespace Service::SSL {
2018-03-23 02:32:50 -04:00
// This is nn::ssl::sf::CertificateFormat
enum class CertificateFormat : u32 {
Pem = 1,
Der = 2,
};
// This is nn::ssl::sf::ContextOption
enum class ContextOption : u32 {
None = 0,
CrlImportDateCheckEnable = 1,
};
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
// This is nn::ssl::Connection::IoMode
enum class IoMode : u32 {
Blocking = 1,
NonBlocking = 2,
};
// This is nn::ssl::sf::OptionType
enum class OptionType : u32 {
DoNotCloseSocket = 0,
GetServerCertChain = 1,
};
// This is nn::ssl::sf::SslVersion
struct SslVersion {
union {
u32 raw{};
BitField<0, 1, u32> tls_auto;
BitField<3, 1, u32> tls_v10;
BitField<4, 1, u32> tls_v11;
BitField<5, 1, u32> tls_v12;
BitField<6, 1, u32> tls_v13;
BitField<24, 7, u32> api_version;
};
};
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
struct SslContextSharedData {
u32 connection_count = 0;
};
class ISslConnection final : public ServiceFramework<ISslConnection> {
public:
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
explicit ISslConnection(Core::System& system_, SslVersion version,
std::shared_ptr<SslContextSharedData>& shared_data,
std::unique_ptr<SSLConnectionBackend>&& backend)
: ServiceFramework{system_, "ISslConnection"}, ssl_version{version},
shared_data{shared_data}, backend{std::move(backend)} {
// clang-format off
static const FunctionInfo functions[] = {
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
{0, &ISslConnection::SetSocketDescriptor, "SetSocketDescriptor"},
{1, &ISslConnection::SetHostName, "SetHostName"},
{2, &ISslConnection::SetVerifyOption, "SetVerifyOption"},
{3, &ISslConnection::SetIoMode, "SetIoMode"},
{4, nullptr, "GetSocketDescriptor"},
{5, nullptr, "GetHostName"},
{6, nullptr, "GetVerifyOption"},
{7, nullptr, "GetIoMode"},
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
{8, &ISslConnection::DoHandshake, "DoHandshake"},
{9, &ISslConnection::DoHandshakeGetServerCert, "DoHandshakeGetServerCert"},
{10, &ISslConnection::Read, "Read"},
{11, &ISslConnection::Write, "Write"},
{12, &ISslConnection::Pending, "Pending"},
{13, nullptr, "Peek"},
{14, nullptr, "Poll"},
{15, nullptr, "GetVerifyCertError"},
{16, nullptr, "GetNeededServerCertBufferSize"},
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
{17, &ISslConnection::SetSessionCacheMode, "SetSessionCacheMode"},
{18, nullptr, "GetSessionCacheMode"},
{19, nullptr, "FlushSessionCache"},
{20, nullptr, "SetRenegotiationMode"},
{21, nullptr, "GetRenegotiationMode"},
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
{22, &ISslConnection::SetOption, "SetOption"},
{23, nullptr, "GetOption"},
{24, nullptr, "GetVerifyCertErrors"},
{25, nullptr, "GetCipherInfo"},
{26, nullptr, "SetNextAlpnProto"},
{27, nullptr, "GetNextAlpnProto"},
{28, nullptr, "SetDtlsSocketDescriptor"},
{29, nullptr, "GetDtlsHandshakeTimeout"},
{30, nullptr, "SetPrivateOption"},
{31, nullptr, "SetSrtpCiphers"},
{32, nullptr, "GetSrtpCipher"},
{33, nullptr, "ExportKeyingMaterial"},
{34, nullptr, "SetIoTimeout"},
{35, nullptr, "GetIoTimeout"},
};
// clang-format on
RegisterHandlers(functions);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
shared_data->connection_count++;
}
~ISslConnection() {
shared_data->connection_count--;
if (fd_to_close.has_value()) {
const s32 fd = *fd_to_close;
if (!do_not_close_socket) {
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
LOG_ERROR(Service_SSL,
"do_not_close_socket was changed after setting socket; is this right?");
} else {
auto bsd = system.ServiceManager().GetService<Service::Sockets::BSD>("bsd:u");
if (bsd) {
auto err = bsd->CloseImpl(fd);
if (err != Service::Sockets::Errno::SUCCESS) {
2023-06-25 22:23:23 -04:00
LOG_ERROR(Service_SSL, "Failed to close duplicated socket: {}", err);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
}
}
}
}
}
private:
SslVersion ssl_version;
std::shared_ptr<SslContextSharedData> shared_data;
std::unique_ptr<SSLConnectionBackend> backend;
std::optional<int> fd_to_close;
bool do_not_close_socket = false;
bool get_server_cert_chain = false;
std::shared_ptr<Network::SocketBase> socket;
bool did_set_host_name = false;
bool did_handshake = false;
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
ResultVal<s32> SetSocketDescriptorImpl(s32 fd) {
LOG_DEBUG(Service_SSL, "called, fd={}", fd);
ASSERT(!did_handshake);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
auto bsd = system.ServiceManager().GetService<Service::Sockets::BSD>("bsd:u");
ASSERT_OR_EXECUTE(bsd, { return ResultInternalError; });
s32 ret_fd;
// Based on https://switchbrew.org/wiki/SSL_services#SetSocketDescriptor
if (do_not_close_socket) {
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
auto res = bsd->DuplicateSocketImpl(fd);
if (!res.has_value()) {
2023-06-25 22:23:23 -04:00
LOG_ERROR(Service_SSL, "Failed to duplicate socket with fd {}", fd);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
return ResultInvalidSocket;
}
fd = *res;
fd_to_close = fd;
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
ret_fd = fd;
} else {
ret_fd = -1;
}
std::optional<std::shared_ptr<Network::SocketBase>> sock = bsd->GetSocket(fd);
if (!sock.has_value()) {
LOG_ERROR(Service_SSL, "invalid socket fd {}", fd);
return ResultInvalidSocket;
}
socket = std::move(*sock);
backend->SetSocket(socket);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
return ret_fd;
}
Result SetHostNameImpl(const std::string& hostname) {
2023-06-25 22:23:23 -04:00
LOG_DEBUG(Service_SSL, "called. hostname={}", hostname);
ASSERT(!did_handshake);
Result res = backend->SetHostName(hostname);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
if (res == ResultSuccess) {
did_set_host_name = true;
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
}
return res;
}
Result SetVerifyOptionImpl(u32 option) {
ASSERT(!did_handshake);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
LOG_WARNING(Service_SSL, "(STUBBED) called. option={}", option);
return ResultSuccess;
}
Result SetIoModeImpl(u32 input_mode) {
auto mode = static_cast<IoMode>(input_mode);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
ASSERT(mode == IoMode::Blocking || mode == IoMode::NonBlocking);
ASSERT_OR_EXECUTE(socket, { return ResultNoSocket; });
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
2023-06-25 22:23:23 -04:00
const bool non_block = mode == IoMode::NonBlocking;
const Network::Errno error = socket->SetNonBlock(non_block);
2023-06-25 22:23:23 -04:00
if (error != Network::Errno::SUCCESS) {
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
LOG_ERROR(Service_SSL, "Failed to set native socket non-block flag to {}", non_block);
}
return ResultSuccess;
}
Result SetSessionCacheModeImpl(u32 mode) {
ASSERT(!did_handshake);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
LOG_WARNING(Service_SSL, "(STUBBED) called. value={}", mode);
return ResultSuccess;
}
Result DoHandshakeImpl() {
ASSERT_OR_EXECUTE(!did_handshake && socket, { return ResultNoSocket; });
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
ASSERT_OR_EXECUTE_MSG(
did_set_host_name, { return ResultInternalError; },
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
"Expected SetHostName before DoHandshake");
Result res = backend->DoHandshake();
did_handshake = res.IsSuccess();
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
return res;
}
std::vector<u8> SerializeServerCerts(const std::vector<std::vector<u8>>& certs) {
struct Header {
u64 magic;
u32 count;
u32 pad;
};
struct EntryHeader {
u32 size;
u32 offset;
};
if (!get_server_cert_chain) {
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
// Just return the first one, unencoded.
ASSERT_OR_EXECUTE_MSG(
!certs.empty(), { return {}; }, "Should be at least one server cert");
return certs[0];
}
std::vector<u8> ret;
Header header{0x4E4D684374726543, static_cast<u32>(certs.size()), 0};
ret.insert(ret.end(), reinterpret_cast<u8*>(&header), reinterpret_cast<u8*>(&header + 1));
size_t data_offset = sizeof(Header) + certs.size() * sizeof(EntryHeader);
for (auto& cert : certs) {
EntryHeader entry_header{static_cast<u32>(cert.size()), static_cast<u32>(data_offset)};
data_offset += cert.size();
ret.insert(ret.end(), reinterpret_cast<u8*>(&entry_header),
reinterpret_cast<u8*>(&entry_header + 1));
}
for (auto& cert : certs) {
ret.insert(ret.end(), cert.begin(), cert.end());
}
return ret;
}
ResultVal<std::vector<u8>> ReadImpl(size_t size) {
ASSERT_OR_EXECUTE(did_handshake, { return ResultInternalError; });
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
std::vector<u8> res(size);
ResultVal<size_t> actual = backend->Read(res);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
if (actual.Failed()) {
return actual.Code();
}
res.resize(*actual);
return res;
}
ResultVal<size_t> WriteImpl(std::span<const u8> data) {
ASSERT_OR_EXECUTE(did_handshake, { return ResultInternalError; });
return backend->Write(data);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
}
ResultVal<s32> PendingImpl() {
LOG_WARNING(Service_SSL, "(STUBBED) called.");
return 0;
}
void SetSocketDescriptor(HLERequestContext& ctx) {
IPC::RequestParser rp{ctx};
const s32 fd = rp.Pop<s32>();
const ResultVal<s32> res = SetSocketDescriptorImpl(fd);
IPC::ResponseBuilder rb{ctx, 3};
rb.Push(res.Code());
rb.Push<s32>(res.ValueOr(-1));
}
void SetHostName(HLERequestContext& ctx) {
const std::string hostname = Common::StringFromBuffer(ctx.ReadBuffer());
const Result res = SetHostNameImpl(hostname);
IPC::ResponseBuilder rb{ctx, 2};
rb.Push(res);
}
void SetVerifyOption(HLERequestContext& ctx) {
IPC::RequestParser rp{ctx};
const u32 option = rp.Pop<u32>();
const Result res = SetVerifyOptionImpl(option);
IPC::ResponseBuilder rb{ctx, 2};
rb.Push(res);
}
void SetIoMode(HLERequestContext& ctx) {
IPC::RequestParser rp{ctx};
const u32 mode = rp.Pop<u32>();
const Result res = SetIoModeImpl(mode);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
IPC::ResponseBuilder rb{ctx, 2};
rb.Push(res);
}
void DoHandshake(HLERequestContext& ctx) {
const Result res = DoHandshakeImpl();
IPC::ResponseBuilder rb{ctx, 2};
rb.Push(res);
}
void DoHandshakeGetServerCert(HLERequestContext& ctx) {
struct OutputParameters {
u32 certs_size;
u32 certs_count;
};
static_assert(sizeof(OutputParameters) == 0x8);
2023-06-25 22:23:23 -04:00
const Result res = DoHandshakeImpl();
OutputParameters out{};
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
if (res == ResultSuccess) {
auto certs = backend->GetServerCerts();
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
if (certs.Succeeded()) {
2023-06-25 22:23:23 -04:00
const std::vector<u8> certs_buf = SerializeServerCerts(*certs);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
ctx.WriteBuffer(certs_buf);
out.certs_count = static_cast<u32>(certs->size());
out.certs_size = static_cast<u32>(certs_buf.size());
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
}
}
IPC::ResponseBuilder rb{ctx, 4};
rb.Push(res);
rb.PushRaw(out);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
}
void Read(HLERequestContext& ctx) {
const ResultVal<std::vector<u8>> res = ReadImpl(ctx.GetWriteBufferSize());
IPC::ResponseBuilder rb{ctx, 3};
rb.Push(res.Code());
if (res.Succeeded()) {
rb.Push(static_cast<u32>(res->size()));
ctx.WriteBuffer(*res);
} else {
rb.Push(static_cast<u32>(0));
}
}
void Write(HLERequestContext& ctx) {
const ResultVal<size_t> res = WriteImpl(ctx.ReadBuffer());
IPC::ResponseBuilder rb{ctx, 3};
rb.Push(res.Code());
rb.Push(static_cast<u32>(res.ValueOr(0)));
}
void Pending(HLERequestContext& ctx) {
const ResultVal<s32> res = PendingImpl();
IPC::ResponseBuilder rb{ctx, 3};
rb.Push(res.Code());
rb.Push<s32>(res.ValueOr(0));
}
void SetSessionCacheMode(HLERequestContext& ctx) {
IPC::RequestParser rp{ctx};
const u32 mode = rp.Pop<u32>();
const Result res = SetSessionCacheModeImpl(mode);
IPC::ResponseBuilder rb{ctx, 2};
rb.Push(res);
}
void SetOption(HLERequestContext& ctx) {
struct Parameters {
OptionType option;
s32 value;
};
static_assert(sizeof(Parameters) == 0x8, "Parameters is an invalid size");
IPC::RequestParser rp{ctx};
const auto parameters = rp.PopRaw<Parameters>();
switch (parameters.option) {
case OptionType::DoNotCloseSocket:
do_not_close_socket = static_cast<bool>(parameters.value);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
break;
case OptionType::GetServerCertChain:
get_server_cert_chain = static_cast<bool>(parameters.value);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
break;
default:
2023-06-25 22:23:23 -04:00
LOG_WARNING(Service_SSL, "Unknown option={}, value={}", parameters.option,
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
parameters.value);
}
IPC::ResponseBuilder rb{ctx, 2};
rb.Push(ResultSuccess);
}
};
class ISslContext final : public ServiceFramework<ISslContext> {
public:
explicit ISslContext(Core::System& system_, SslVersion version)
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
: ServiceFramework{system_, "ISslContext"}, ssl_version{version},
shared_data{std::make_shared<SslContextSharedData>()} {
static const FunctionInfo functions[] = {
{0, &ISslContext::SetOption, "SetOption"},
{1, nullptr, "GetOption"},
{2, &ISslContext::CreateConnection, "CreateConnection"},
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
{3, &ISslContext::GetConnectionCount, "GetConnectionCount"},
{4, &ISslContext::ImportServerPki, "ImportServerPki"},
{5, &ISslContext::ImportClientPki, "ImportClientPki"},
{6, nullptr, "RemoveServerPki"},
{7, nullptr, "RemoveClientPki"},
{8, nullptr, "RegisterInternalPki"},
{9, nullptr, "AddPolicyOid"},
{10, nullptr, "ImportCrl"},
{11, nullptr, "RemoveCrl"},
{12, nullptr, "ImportClientCertKeyPki"},
{13, nullptr, "GeneratePrivateKeyAndCert"},
};
RegisterHandlers(functions);
}
private:
SslVersion ssl_version;
std::shared_ptr<SslContextSharedData> shared_data;
2023-02-19 14:42:12 -05:00
void SetOption(HLERequestContext& ctx) {
struct Parameters {
ContextOption option;
s32 value;
};
static_assert(sizeof(Parameters) == 0x8, "Parameters is an invalid size");
IPC::RequestParser rp{ctx};
const auto parameters = rp.PopRaw<Parameters>();
LOG_WARNING(Service_SSL, "(STUBBED) called. option={}, value={}", parameters.option,
parameters.value);
IPC::ResponseBuilder rb{ctx, 2};
rb.Push(ResultSuccess);
}
2023-02-19 14:42:12 -05:00
void CreateConnection(HLERequestContext& ctx) {
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
LOG_WARNING(Service_SSL, "called");
auto backend_res = CreateSSLConnectionBackend();
IPC::ResponseBuilder rb{ctx, 2, 0, 1};
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
rb.Push(backend_res.Code());
if (backend_res.Succeeded()) {
rb.PushIpcInterface<ISslConnection>(system, ssl_version, shared_data,
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
std::move(*backend_res));
}
}
void GetConnectionCount(HLERequestContext& ctx) {
LOG_DEBUG(Service_SSL, "connection_count={}", shared_data->connection_count);
Implement SSL service This implements some missing network APIs including a large chunk of the SSL service, enough for Mario Maker (with an appropriate mod applied) to connect to the fan server [Open Course World](https://opencourse.world/). Connecting to first-party servers is out of scope of this PR and is a minefield I'd rather not step into. ## TLS TLS is implemented with multiple backends depending on the system's 'native' TLS library. Currently there are two backends: Schannel for Windows, and OpenSSL for Linux. (In reality Linux is a bit of a free-for-all where there's no one 'native' library, but OpenSSL is the closest it gets.) On macOS the 'native' library is SecureTransport but that isn't implemented in this PR. (Instead, all non-Windows OSes will use OpenSSL unless disabled with `-DENABLE_OPENSSL=OFF`.) Why have multiple backends instead of just using a single library, especially given that Yuzu already embeds mbedtls for cryptographic algorithms? Well, I tried implementing this on mbedtls first, but the problem is TLS policies - mainly trusted certificate policies, and to a lesser extent trusted algorithms, SSL versions, etc. ...In practice, the chance that someone is going to conduct a man-in-the-middle attack on a third-party game server is pretty low, but I'm a security nerd so I like to do the right security things. My base assumption is that we want to use the host system's TLS policies. An alternative would be to more closely emulate the Switch's TLS implementation (which is based on NSS). But for one thing, I don't feel like reverse engineering it. And I'd argue that for third-party servers such as Open Course World, it's theoretically preferable to use the system's policies rather than the Switch's, for two reasons 1. Someday the Switch will stop being updated, and the trusted cert list, algorithms, etc. will start to go stale, but users will still want to connect to third-party servers, and there's no reason they shouldn't have up-to-date security when doing so. At that point, homebrew users on actual hardware may patch the TLS implementation, but for emulators it's simpler to just use the host's stack. 2. Also, it's good to respect any custom certificate policies the user may have added systemwide. For example, they may have added custom trusted CAs in order to use TLS debugging tools or pass through corporate MitM middleboxes. Or they may have removed some CAs that are normally trusted out of paranoia. Note that this policy wouldn't work as-is for connecting to first-party servers, because some of them serve certificates based on Nintendo's own CA rather than a publicly trusted one. However, this could probably be solved easily by using appropriate APIs to adding Nintendo's CA as an alternate trusted cert for Yuzu's connections. That is not implemented in this PR because, again, first-party servers are out of scope. (If anything I'd rather have an option to _block_ connections to Nintendo servers, but that's not implemented here.) To use the host's TLS policies, there are three theoretical options: a) Import the host's trusted certificate list into a cross-platform TLS library (presumably mbedtls). b) Use the native TLS library to verify certificates but use a cross-platform TLS library for everything else. c) Use the native TLS library for everything. Two problems with option a). First, importing the trusted certificate list at minimum requires a bunch of platform-specific code, which mbedtls does not have built in. Interestingly, OpenSSL recently gained the ability to import the Windows certificate trust store... but that leads to the second problem, which is that a list of trusted certificates is [not expressive enough](https://bugs.archlinux.org/task/41909) to express a modern certificate trust policy. For example, Windows has the concept of [explicitly distrusted certificates](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn265983(v=ws.11)), and macOS requires Certificate Transparency validation for some certificates with complex rules for when it's required. Option b) (using native library just to verify certs) is probably feasible, but it would miss aspects of TLS policy other than trusted certs (like allowed algorithms), and in any case it might well require writing more code, not less, compared to using the native library for everything. So I ended up at option c), using the native library for everything. What I'd *really* prefer would be to use a third-party library that does option c) for me. Rust has a good library for this, [native-tls](https://docs.rs/native-tls/latest/native_tls/). I did search, but I couldn't find a good option in the C or C++ ecosystem, at least not any that wasn't part of some much larger framework. I was surprised - isn't this a pretty common use case? Well, many applications only need TLS for HTTPS, and they can use libcurl, which has a TLS abstraction layer internally but doesn't expose it. Other applications only support a single TLS library, or use one of the aforementioned larger frameworks, or are platform-specific to begin with, or of course are written in a non-C/C++ language, most of which have some canonical choice for TLS. But there are also many applications that have a set of TLS backends just like this; it's just that nobody has gone ahead and abstracted the pattern into a library, at least not a widespread one. Amusingly, there is one TLS abstraction layer that Yuzu already bundles: the one in ffmpeg. But it is missing some features that would be needed to use it here (like reusing an existing socket rather than managing the socket itself). Though, that does mean that the wiki's build instructions for Linux (and macOS for some reason?) already recommend installing OpenSSL, so no need to update those. ## Other APIs implemented - Sockets: - GetSockOpt(`SO_ERROR`) - SetSockOpt(`SO_NOSIGPIPE`) (stub, I have no idea what this does on Switch) - `DuplicateSocket` (because the SSL sysmodule calls it internally) - More `PollEvents` values - NSD: - `Resolve` and `ResolveEx` (stub, good enough for Open Course World and probably most third-party servers, but not first-party) - SFDNSRES: - `GetHostByNameRequest` and `GetHostByNameRequestWithOptions` - `ResolverSetOptionRequest` (stub) ## Fixes - Parts of the socket code were previously allocating a `sockaddr` object on the stack when calling functions that take a `sockaddr*` (e.g. `accept`). This might seem like the right thing to do to avoid illegal aliasing, but in fact `sockaddr` is not guaranteed to be large enough to hold any particular type of address, only the header. This worked in practice because in practice `sockaddr` is the same size as `sockaddr_in`, but it's not how the API is meant to be used. I changed this to allocate an `sockaddr_in` on the stack and `reinterpret_cast` it. I could try to do something cleverer with `aligned_storage`, but casting is the idiomatic way to use these particular APIs, so it's really the system's responsibility to avoid any aliasing issues. - I rewrote most of the `GetAddrInfoRequest[WithOptions]` implementation. The old implementation invoked the host's getaddrinfo directly from sfdnsres.cpp, and directly passed through the host's socket type, protocol, etc. values rather than looking up the corresponding constants on the Switch. To be fair, these constants don't tend to actually vary across systems, but still... I added a wrapper for `getaddrinfo` in `internal_network/network.cpp` similar to the ones for other socket APIs, and changed the `GetAddrInfoRequest` implementation to use it. While I was at it, I rewrote the serialization to use the same approach I used to implement `GetHostByNameRequest`, because it reduces the number of size calculations. While doing so I removed `AF_INET6` support because the Switch doesn't support IPv6; it might be nice to support IPv6 anyway, but that would have to apply to all of the socket APIs. I also corrected the IPC wrappers for `GetAddrInfoRequest` and `GetAddrInfoRequestWithOptions` based on reverse engineering and hardware testing. Every call to `GetAddrInfoRequestWithOptions` returns *four* different error codes (IPC status, getaddrinfo error code, netdb error code, and errno), and `GetAddrInfoRequest` returns three of those but in a different order, and it doesn't really matter but the existing implementation was a bit off, as I discovered while testing `GetHostByNameRequest`. - The new serialization code is based on two simple helper functions: ```cpp template <typename T> static void Append(std::vector<u8>& vec, T t); void AppendNulTerminated(std::vector<u8>& vec, std::string_view str); ``` I was thinking there must be existing functions somewhere that assist with serialization/deserialization of binary data, but all I could find was the helper methods in `IOFile` and `HLERequestContext`, not anything that could be used with a generic byte buffer. If I'm not missing something, then maybe I should move the above functions to a new header in `common`... right now they're just sitting in `sfdnsres.cpp` where they're used. - Not a fix, but `SocketBase::Recv`/`Send` is changed to use `std::span<u8>` rather than `std::vector<u8>&` to avoid needing to copy the data to/from a vector when those methods are called from the TLS implementation.
2023-06-19 21:17:43 -04:00
IPC::ResponseBuilder rb{ctx, 3};
rb.Push(ResultSuccess);
rb.Push(shared_data->connection_count);
}
2023-02-19 14:42:12 -05:00
void ImportServerPki(HLERequestContext& ctx) {
IPC::RequestParser rp{ctx};
const auto certificate_format = rp.PopEnum<CertificateFormat>();
[[maybe_unused]] const auto pkcs_12_certificates = ctx.ReadBuffer(0);
constexpr u64 server_id = 0;
LOG_WARNING(Service_SSL, "(STUBBED) called, certificate_format={}", certificate_format);
IPC::ResponseBuilder rb{ctx, 4};
rb.Push(ResultSuccess);
rb.Push(server_id);
}
2023-02-19 14:42:12 -05:00
void ImportClientPki(HLERequestContext& ctx) {
[[maybe_unused]] const auto pkcs_12_certificate = ctx.ReadBuffer(0);
[[maybe_unused]] const auto ascii_password = [&ctx] {
if (ctx.CanReadBuffer(1)) {
return ctx.ReadBuffer(1);
}
return std::span<const u8>{};
}();
constexpr u64 client_id = 0;
LOG_WARNING(Service_SSL, "(STUBBED) called");
IPC::ResponseBuilder rb{ctx, 4};
rb.Push(ResultSuccess);
rb.Push(client_id);
}
};
class ISslService final : public ServiceFramework<ISslService> {
public:
explicit ISslService(Core::System& system_) : ServiceFramework{system_, "ssl"} {
// clang-format off
static const FunctionInfo functions[] = {
{0, &ISslService::CreateContext, "CreateContext"},
{1, nullptr, "GetContextCount"},
{2, nullptr, "GetCertificates"},
{3, nullptr, "GetCertificateBufSize"},
{4, nullptr, "DebugIoctl"},
{5, &ISslService::SetInterfaceVersion, "SetInterfaceVersion"},
{6, nullptr, "FlushSessionCache"},
{7, nullptr, "SetDebugOption"},
{8, nullptr, "GetDebugOption"},
{8, nullptr, "ClearTls12FallbackFlag"},
};
// clang-format on
RegisterHandlers(functions);
}
private:
2023-02-19 14:42:12 -05:00
void CreateContext(HLERequestContext& ctx) {
struct Parameters {
SslVersion ssl_version;
INSERT_PADDING_BYTES(0x4);
u64 pid_placeholder;
};
static_assert(sizeof(Parameters) == 0x10, "Parameters is an invalid size");
IPC::RequestParser rp{ctx};
const auto parameters = rp.PopRaw<Parameters>();
LOG_WARNING(Service_SSL, "(STUBBED) called, api_version={}, pid_placeholder={}",
parameters.ssl_version.api_version, parameters.pid_placeholder);
2018-03-23 02:32:50 -04:00
IPC::ResponseBuilder rb{ctx, 2, 0, 1};
rb.Push(ResultSuccess);
rb.PushIpcInterface<ISslContext>(system, parameters.ssl_version);
}
2023-02-19 14:42:12 -05:00
void SetInterfaceVersion(HLERequestContext& ctx) {
IPC::RequestParser rp{ctx};
u32 ssl_version = rp.Pop<u32>();
LOG_DEBUG(Service_SSL, "called, ssl_version={}", ssl_version);
IPC::ResponseBuilder rb{ctx, 2};
rb.Push(ResultSuccess);
}
};
void LoopProcess(Core::System& system) {
auto server_manager = std::make_unique<ServerManager>(system);
server_manager->RegisterNamedService("ssl", std::make_shared<ISslService>(system));
ServerManager::RunServer(std::move(server_manager));
2018-03-23 02:32:50 -04:00
}
} // namespace Service::SSL