Warning message

Log in to add comments.

Latest Posts

  • Changes coming to TLS: Part One

    Authored by: Huzaifa Sidhpurwala

    Transport layer Security version 1.3 (TLS 1.3) is the latest version of the SSL/TLS protocol which is currently under development by the IETF. It offers several security and performance improvements as compared to the previous versions. While there are several technical resouces which discuss the finer aspects of this new protocol, this two-part article is a quick reference to new features and major changes in the TLS protocol.

    Faster Handshakes

    In TLS, before data can be encrypted, a secure channel needs to created. This is achieved via the handshake protocol in which a cipher suite is negotiated between the client and the server and key share materials are exchanged.

    The handshake is initiated by the TLS client utilizing a ClientHello message sent to the server. This message contains, among other things, a list of cipher suites which the client is capable of supporting. The server replies to this message in its ServerHello message by picking one of the cipher suites from the client list and, along with it, the server also sends its key share and the site's certificate to the client.

    The client receives all of the above, generates its own key share, combines it with the server key share and generates bulk encryption keys for the session. The client then sends the server its key share and sends a FINISHED message which contains a signed hash of all the data which was previously exchanged. The server does the same thing by sending its own FINISHED message.

    TLS 1.3 handshake

    This ends the handshake and results in a cipher suite and a bulk encryption key being negotiated between the client and the server and takes two full rounds of data exchange.1

    TLS 1.3 aims to make this faster by reducing the handshake to just one round trip (1-RTT). In TLS 1.3 the ClientHello not only contains a list of supported ciphers, but also it makes a guess as to what key agreement protocol the server is likely to chose and sends a key share for that particular protocol.

    TLS 1.3 handshake

    As soon as the server selects the key agreement protocol it sends its own key share and, at the same time, generates the bulk encryption key (since it already has the client key share). In doing so, the computers can switch to encrypted messages one whole round trip in advance. The ServerHello also contains the FINISHED message. The client receives the server key share, generates bulk encryption keys, sends the FINISHED message and is immediately ready to send encrypted data. This results in faster handshakes and better performance.2

    Faster Session Resumptions

    The TLS protocol has a provision for session resumption. The general idea is to avoid a full handshake by storing the secret information of previous sessions and reusing those when connecting to a host the next time. This drastically reduces latency and CPU usage. However in modern times the session resumption is usually frowned upon because it can easily compromise Perfect Forward Secrecy (PFS).

    Session resumption can be achieved in one of the two following ways:

    1. Session ID:
      In a full handshake the server sends a Session ID as part of the ServerHello message. On a subsequent connection the client can use this session ID and pass it to the server when connecting. Because both server and client have saved the last session’s “secret state” under the session ID they can simply resume the TLS session where they left off.

    2. Session Tickets:
      The second mechanism to resume a TLS session are Session Tickets. This extension transmits the server’s secret state to the client, encrypted with a key only known to the server. That ticket key is protecting the TLS connection now and in the future and is the weak spot an attacker will target. The client will store its secret information for a TLS session along with the ticket received from the server. By transmitting that ticket back to the server at the beginning of the next TLS connection both parties can resume their previous session, given that the server can still access the secret key that was used to encrypt it.

    In both of the above methods session resumption is done by using 1-RTT (one round trip). However TLS 1.3 achieves this by using 0-RTT. When a TLS 1.3 client connects to a server, they agree on a session resumption key. If pre-shared keys (PSK) are used the server provides a Session Ticket which can be encrypted using the PSK if required.

    During Session resumption, the client sends the Session Ticket in the ClientHello and then immediately, without waiting for the RTT to complete, sends the encrypted data. The server figures out the PSK from the session ticket and uses it to decrypt the data and resume the connection. The client may also send a key share, so that the server and the client can switch to a fresh bulk encryption key for the rest of the session.

    TLS 1.3 handshake

    There are two possible problems with the above implementation though:

    1. No PFS when using Session Resumption:
      Since the PSK is not agreed upon using fresh Diffie Hellman, it does not provide Forward Secrecy against a compromise of the session key. This problem can be solved by rotating the session keys regularly.

    2. This can easily lead to replay attacks:
      Since the 0-RTT data does not have any state information associated with it, an attacker could easily resend the 0-RTT message, and the server will decrypt the associated data and act on the data it received. One way to avoid this is to make sure that the server does not perform any "important action" (like transferring secret information or making a financial transaction) based on the 0-RTT data alone. For such operations servers can impose 1-RTT session resumptions.

    DTLS has an additional concern: during re-handshake it is not known whether the peer is alive at all. Data sent along with the handshake message may have ended up in a black hole, going no where.

    Part 2 of the blog discusses the various cryptographic changes found in TLS 1.3 and their implications.


    1. TLS 1.2 has an extension called "false start", which can reduce the initial handshake to 1-RTT, it bundles the client application data with its FINISHED message. 

    2. The roundtrips mentioned are TLS protocol roundtrips, not actual network roundtrips. In reality you have additional roundtrips due to TCP establishment. Most implementations today provide a way to use TCP fast open to reduce that additional round-trip. 

    Posted: 2017-03-29T13:30:00+00:00
  • Customer security awareness: alerting you to vulnerabilities that are of real risk

    Authored by: Christopher Robinson

    Every day we are bombarded with information. Something is always happening somewhere to someone and unfortunately it's rarely good. Looking at this through the lens of information security, NOT getting the right details at the appropriate time could be the difference from stopping and blocking an attack, or being the next sad, tragic headline...

    Red Hat Product Security oversees the vulnerability remediation for all of Red Hat's products. Our dual mission of governing guidelines and standards for how our products are composed and delivered is balanced with our in-taking, assessing, and responding to information about security vulnerabilities that might impact those products. Once a flaw has been identified, part of our role is to understand its real impact and try to produce a calm, clear direction to get issues that matter remediated. One big challenge is understanding when something is bad and could cause harm compared with something that is completely terrible and WILL cause major havoc out “in the wild." For the layperson, the facts and the hype can be extremely difficult and time-consuming to separate so that they can act appropriately.

    Recent trends in the security field haven't been helping. It seems as if every month there is a new bug that has a cute name, a logo, and a webstore selling stickers and stuffed animals. While awareness of a problem is an excellent goal, oftentimes the flashing blinky text and images obscure how bad (or not) an issue is.

    Thankfully, for over 15 years Red Hat Product Security has been providing calm, accurate, timely advice around these types of issues. We're able to separate the hope from the hype, so to speak. To that end, with the meteoric rise of “branded” flaws not stopping in the foreseeable future, Red Hat Product Security developed a special process to help inform our valued subscribers and partners when these situations arise. We call it our Customer Security Awareness (CSAw) process:

    We've augmented our processes to include enhanced oversight and handling of these very special issues. Some of these issues could be of such grave risk the need for quick actions and good advice merits extra special handling. Other times we might recognize that a security bug has the potential to have it's own PR agent, we take the right steps so that customers proactively get the appropriate level of information, allowing them to decide how quickly they need to react based on their own risk appetites. We ensure we provide special tools and extra alerts so that when these things really DO matter, the decision makers have the right data to move forward.

    For more details about the process, please check out the Red Hat Product Security Center or reach out to us via secalert@redhat.com or our Twitter Account @RedHatSecurity.

    Posted: 2017-03-22T13:30:00+00:00
  • Red Hat Product Security Risk Report 2016

    Authored by: Red Hat Product...

    At Red Hat, our dedicated Product Security team analyzes threats and vulnerabilities against all our products and provides relevant advice and updates through the Red Hat Customer Portal. Customers can rely on this expertise to help them quickly address the issues that can cause high risks and avoid wasting time or effort on those that don’t.

    Red Hat delivers certified, signed, supported versions of the open source solutions that enable cost-effective innovation for the enterprise. This is the Red Hat value chain.

    This report explores the state of security risk for Red HatⓇ products for calendar year 2016. We look at key metrics, specific vulnerabilities, and the most common ways that security issues affected users of Red Hat products.

    Among our findings:

    • Looking only at issues affecting base Red Hat Enterprise LinuxⓇ releases, we released 38 Critical security advisories addressing 50 Critical vulnerabilities. Of those issues, 100% had fixes the same or next day after the issue was public.

    • During that same timeframe, across the whole Red Hat portfolio, 76% of Critical issues had updates to address them the same or next day after the issue was public with 98% addressed within a week of the issue being public.

    • A catchy name or a flashy headline for a vulnerability doesn't tell much about its risk. The Red Hat Product Security Team helps customers determine a vulnerability’s actual impact. Most 2016 issues that mattered were not branded.

    Vulnerabilities

    Across all Red Hat products, and for all issue severities, we fixed more than 1,300 vulnerabilities1 by releasing more than 600 security advisories in 2016. Critical2 vulnerabilities pose the most risk to an organization. Most Critical vulnerabilities occur in browser or browser components, so Red Hat Enterprise Linux server installations will generally be affected by far fewer critical vulnerabilities. One way customers can reduce risk when using our modular products is to make sure they install the right variant and review the package set to remove packages they don’t need.

    The Red Hat value chain

    Red Hat products are based on open source software. Some Red Hat products contain several thousand individual packages, each of which is based on separate, third-party software from upstream projects.

    Red Hat engineers play a part in many upstream components, but handling and managing vulnerabilities across thousands of third-party components is a significant task. Red Hat has a dedicated Product Security team that monitors issues affecting Red Hat products and works closely with upstream projects.

    For more than 15 years, Red Hat Product Security has been a recognized leader in fixing security flaws across the Linux stack. In 2016, we investigated more than 2,600 vulnerabilities that potentially affected parts of our products, leading to fixes for 1,346 vulnerabilities. That’s a 30% increase over 2015, when the team investigated 2,000.

    Vulnerabilities known to Red Hat in advance of being made public are known as “under embargo.” Unlike companies shipping proprietary software, Red Hat is not in sole control of the date each flaw is made public. This is a good thing, as it leads to much shorter times between when a flaw is first reported and when it becomes public. Shorter embargo periods make flaws much less valuable to attackers. They know a flaw in open source is likely to get fixed quickly, shortening their window of opportunity to exploit it.

    For 2016, across all products, we knew about 394 (29%) of the vulnerabilities we addressed before making them public, down slightly from 32% in 2015. We expect this figure to vary from year to year. Across all products and vulnerabilities of all severities known to us in advance, the median embargo was seven days. This is much lower than 2015, when the median embargo was 13 days.

    Figure 2: Red Hat Product Security monitors multiple sources to identify vulnerabilities. The value of your Red Hat subscription at work.

    The full report is available for download.


    1. Red Hat Product Security assigns a Common Vulnerabilities and Exposures (CVE) name to every security issue we fix. In this report, we equate vulnerabilities to CVEs. 

    2. Red Hat rates vulnerabilities on a four-point scale that shows at a glance how much concern Red Hat has about each security issue. The scale rates vulnerabilities as Low, Moderate, Important, or Critical. By definition, a Critical vulnerability is one that could be exploited remotely and automatically by a worm. However we, like other vendors, also stretch the definition to include those flaws that affect web browsers or plug-ins where a user only needs to visit a malicious (or compromised) website to be exploited. 

    Posted: 2017-03-07T14:39:02+00:00
  • How Threat Modeling Helps Discover Security Vulnerabilities

    Authored by: Hooman Broujerdi

    Application threat modeling can be used as an approach to secure software development, as it is a nice preventative measure for dealing with security issues, and mitigates the time and effort required to deal with vulnerabilities that may arise later throughout the application's production life cycle. Unfortunately, it seems security has no place in the development life cycle, however, while CVE bug tracking databases and hacking incident reports proves that it ought to be. Some of the factors that seem to have contributed as to why there's a trend of insecure software development are:

    a) Iron Triangle Constraint: the relationship between time, resources, and budget. From a management standpoint there's an absolute need for the resources (people) to have appropriate skills to be able to implement the software business problem. Unfortunately, resources are not always available and are an expensive factor to consider. Additionally, the time required to produce quality software that solves the business problem is always an intensive challenge, not to mention that constraints in the budget seem to have always been a rigid requirement for any development team.

    b) Security as an Afterthought: taking security for granted has an adverse effect on producing a successful piece of software. Software engineers and managers tend to focus on delivering the actual business requirements and closing the gap between when the business idea is born and when the software has actually hit the market. This creates a mindset that security does not add any business value and it can always be added on rather than built into the software.

    c) Security vs Usability: another reason that seems to be a showstopper in a secure software delivery process is the idea that security makes the software usability more complex and less intuitive (e.g. security configuration is often too complicated to manage). It is absolutely true that the incorporation of security comes with a cost. Psychological Acceptability should be recognized as a factor, but not to the extent of ruling out security as part of a software development life cycle.

    With a and b being the main factors for not adopting security into the Software Development Life Cycle (SDLC), development without bringing security in the early stages turns out to have disastrous consequences. Many vulnerabilities go undetected allowing hackers to penetrate the applications and cause damage and, in the end, harm the reputations of the companies using the software as well as those developing it.

    What is Threat Modeling?

    Threat modeling is a systematic approach for developing resilient software. It identifies the security objective of the software, threats to it, and vulnerabilities in the application being developed. It will also provide insight into an attacker's perspective by looking into some of the entry and exit points that attackers are looking for in order to exploit the software.

    Challenges

    Although threat modeling appears to have proven useful for eliminating security vulnerabilities, it seems to have added a challenge to the overall process due to the gap between security engineers and software developers. Because security engineers are usually not involved in the design and development of the software, it often becomes a time consuming effort to embark on brainstorming sessions with other engineers to understand the specific behavior, and define all system components of the software specifically as the application gets complex.

    Legacy Systems

    While it is important to model threats to a software application in the project life cycle, it is particularly important to threat model legacy software because there's a high chance that the software was originally developed without threat models and security in mind. This is a real challenge as legacy software tends to lack detailed documentation. This, specifically, is the case with open source projects where a lot of people contribute, adding notes and documents, but they may not be organized; consequently making threat modeling a difficult task.

    Threat Modeling Crash Course

    Threat modeling can be drilled down to three steps: characterizing the software, identifying assets and access points, and identifying threats.

    Characterizing the Software

    At the start of the process the system in question needs to be thoroughly understood. This includes reviewing the correlation of every single component as well as defining the usage scenarios and dependencies. This is a critical step to understanding the underlying architecture and implementation details of the system. The information from this process is used to produce a data flow diagram (DFD) which provides the best representation for identifying different security zones where data will be in transit or stored.

    Data Flow Diagram for a typical web application

    Depending on the type and complexity of the system, this phase may also be drilled down into more detailed diagrams that could be used to help understand the system better, and ultimately address a broader range of potential threats.

    Identifying Assets and Access Points

    The next phase of the threat modeling exercise is where assets and access points of the system need to be clearly identified. System assets are the components that need to be protected against misuse by an attacker. Assets could be tangible such as configuration files, sensitive information, and processes or could potentially be an abstract concept like data consistency. Access points, also known as attack surfaces, are the path adversaries use to access the targeted endpoint. This could be an open port or protocol, file system read and write privileges, or authentication mechanism. Once the assets and access points are identified, a data access control matrix can be generated and the access level privilege for each entity can be defined.

    Data Access Control Matrix

    Identifying Threats

    Given the first two phases are complete, specific threats to the system can be identified. Using one of the systematic approaches towards the threat identification process can help organize the effort. The primary approaches are: attack tree based approach, stochastic model based approaches, and categorized threat lists.

    Attack trees have been used widely to identify threats, but categorized lists seem to be more comprehensive and easier to use. Some implementations are Microsoft's STRIDE, OWASP's Top 10 Vulnerabilities, and CWE/SANS' Top 25 Most Dangerous Software Errors.

    Threat identification using OWASP top 10 vulnerabilities

    Although the stochastic based approach is outside the scope of this writing, additional information is available for download.

    The key to generating successful and comprehensive threat lists against external attacks relies heavily on the accuracy of the system architecture model and the corresponding DFD that's been created. These are the means to identify the behavior of the system for each component, and to determine whether a vulnerability exists as a result.

    Risk Ranking

    Calculating the risk of each relevant threat associated with the software is the next step in the process. There are a number of different ways to calculate this risk however OWASP has already documented a methodology which can be used for threat prioritization. The crux of this method is to determine the severity of the risk associated with each threat and come up with a weighting factor to address each identified threat, depending on the significance of the issue to the business. It is also important to understand that threat modeling has to be revisited occasionally to ensure it has not become outdated.

    Mitigation and Control

    Threats selected from previous steps now need to be mitigated. Security engineers should provide a series of countermeasures to ensure that all security aspects of the issues are addressed by developers during the development process. A critical point at this stage is to ensure that the security implementation cost does not exceed the expected risk. The mitigation scope has to be clearly defined to ensure that meaningful security efforts align with the organization's security vision.

    Threat Analysis/Verification

    Threat analysis and verification focuses on the security delivery, after the code development and testing has started. This is a key step towards hardening the software against attacks and threats that were identified earlier. Usually a threat model owner is involved during the process to ensure relevant discussions are had on each remedy implementation and whether the priority of a specific threat can be re-evaluated.

    How Threat Modeling Integrates into SDLC

    Until now the identified threats to the system used to allow the engineering team to make better, informed decisions during the software development life cycle. Continuous integration and delivery being the key to agile development practices make any extra paper work a drag on the development flow. This also resurrects the blocking issues - Iron Triangle Constraint - mentioned earlier which might delay the release as a result. Therefore, it is essential to automate the overall threat modeling process into the continuous delivery pipeline, to ensure security is enforced during the early stages of product development.

    Although there is no one-size-fits-all approach to automating threat modeling into SDLC, and any threat modeling automation technique has to address specific security integration problems, there are various automation implementations available, including Threat Modeling with Architectural Risk Patterns - AppSec USA 2016, Developer-Driven Threat Modeling, and Automated Threat Modeling through the Software Development Life-Cycle that could help integrating threat modeling into the software delivery process.

    Should you use Threat Modeling anyway?

    Threat modeling not only adds value to company's product but also encourages a security posture for its products by providing a holistic security view of the system components. This is used as a security baseline where any development effort can have a clear vision of what needs to be done to ensure security requirements are met with the company benefiting in the long term.

    Posted: 2017-03-02T21:00:00+00:00
  • Debugging a kernel in QEMU/libvirt - Part II

    Authored by: Wade Mealing

    This blog has previously shown how to configure a Red Hat Enterprise Linux system for kernel debugging, it expects that the system has been configured, have the source code matching the installed kernel version handy, and the reader is ready to follow along.

    This should not be running on a productions system as system function interruption is guaranteed.

    The particular problem that will be investigated is CVE-2016-9793. As discussed on the Oss-security list, this vulnerability was classified as an integer overflow, which must be addressed.

    Eric Dumazet describes the patch as (taken from the commit that attempts to fix the flaw):

    $ git show b98b0bc8c431e3ceb4b26b0dfc8db509518fb290
    commit b98b0bc8c431e3ceb4b26b0dfc8db509518fb290
    Author: Eric Dumazet <edumazet@google.com>
    Date:   Fri Dec 2 09:44:53 2016 -0800
    
        net: avoid signed overflows for SO_{SND|RCV}BUFFORCE
    
        CAP_NET_ADMIN users should not be allowed to set negative
        sk_sndbuf or sk_rcvbuf values, as it can lead to various memory
        corruptions, crashes, OOM...
    
        Note that before commit 82981930125a ("net: cleanups in
        sock_setsockopt()"), the bug was even more serious, since SO_SNDBUF
        and SO_RCVBUF were vulnerable.
    
        This needs to be backported to all known linux kernels.
    
        Again, many thanks to syzkaller team for discovering this gem.
    
        Signed-off-by: Eric Dumazet <edumazet@google.com>
        Reported-by: Andrey Konovalov <andreyknvl@google.com>
        Signed-off-by: David S.  Miller <davem@davemloft.net>
    
    diff --git a/net/core/sock.c b/net/core/sock.c
    index 5e3ca41..00a074d 100644
    --- a/net/core/sock.c
    +++ b/net/core/sock.c
    @@ -715,7 +715,7 @@ int sock_setsockopt(struct socket *sock, int level, int optname,
                    val = min_t(u32, val, sysctl_wmem_max);
     set_sndbuf:
                    sk->sk_userlocks |= SOCK_SNDBUF_LOCK;
    -               sk->sk_sndbuf = max_t(u32, val * 2, SOCK_MIN_SNDBUF);
    +               sk->sk_sndbuf = max_t(int, val * 2, SOCK_MIN_SNDBUF);
                    /* Wake up sending tasks if we upped the value.  */
                    sk->sk_write_space(sk);
                    break;
    @@ -751,7 +751,7 @@ set_rcvbuf:
                     * returning the value we actually used in getsockopt
                     * is the most desirable behavior.
                     */
    -               sk->sk_rcvbuf = max_t(u32, val * 2, SOCK_MIN_RCVBUF);
    +               sk->sk_rcvbuf = max_t(int, val * 2, SOCK_MIN_RCVBUF);
                    break;
    
            case SO_RCVBUFFORCE:
    

    The purpose of the investigation is to determine if this flaw affects the shipped kernels.

    User interaction with the kernel happen through syscalls and ioctls. In this case, the issue is the setsockopt syscall. This specific call ends up being handled in a function named sock_setsockopt as shown in the patch.
    The flaw is not always clearly documented in patches, but in this case the area that the patch modifies is an ideal place to start looking.

    Investigating sock_setsockopt function

    The sock_setsockopt code shown below has the relevant parts highlighted in an attempt to explain key concepts that complicate the investigation of this flaw.

    A capabilities check is the first step that must be overcome when attempting to force set the snd_buff size. Inspecting the function sock_setsockopt code, a capable() check enforces the process has CAP_NET_ADMIN privilege to force buffer sizes. The attack vector is reduced by requiring this capability but not entirely mitigated. The root user by default has these capabilities, and it can be granted to binaries that run by other users. The relevant section of code is:

             if (!capable(CAP_NET_ADMIN)) {
                 ret = -EPERM;
                 break;
              }
    

    The reproducer would need to have CAP_NET_ADMIN capabilities/permissions to run setsockopt() with the SO_RCVBUFFORCE parameter. To read more about Linux kernel capabilities checkout the setcap (8) man page and the capabilities (7) man page.

    We can see from the patch and surrounding discussion that it is possible to set the size of sk->sndbuf to be negative. Following the flow of code, it would then enter the max_t macro before being assigned. The patch explicitly changes the max_t macros type to be cast.

    Using the GDB debugger and setting a breakpoint will show how various sizes values affect the final value of sk->sndbuf.

    Integer overflows

    The patch shows that the type used in the max_t macro compare was changed from u32 (unsigned 32 bit integer) to int (signed 32 bit integer). Before we make assumptions or do any kind of investigation, we can make a hypothesis that the problem exists with the outcome of the max_t.

    Here is the definition of max_t:

    #define max_t(type, x, y) ({            \
        type __max1 = (x);            \
        type __max2 = (y);            \
        __max1 > __max2 ? __max1: __max2; })
    

    My understanding of the max_t macro is that it would cast both the second and third parameters to the type specified by the first parameter returning __max1 if __max1 was greater than __max2. The unintended side affect would be that when casting to an unsigned type the comparison would turn negative values into large integer values.

    It may be tempting to program the relevant macro, type definitions, and operations on the variables into a small C program to test. Resist! Armed with your kernel debugging knowledge and a small C program to exercise the code, we can see how the tool-chain decided to create this code.

    For the test case, we'll need to consider using values that will test how the compiler and architecture deals with these kind of overflows. Input that would could create overflows or final negative values should be used as test cases.

    Building the test case

    To exercise this particular section of code (before the patch) we can build a small reproducer in C. Feel free to choose a language and write your test code in which you can set the socket options with the same way.

    #include <stdio.h>
    #include <limits.h>
    #include <linux/types.h>
    #include <sys/types.h>
    
    #include <sys/socket.h>
    #include <stdio.h>
    #include <error.h>
    #include <errno.h>
    #include <string.h>
    
    int main(int argc, char **argv)
    {
        int sockfd, sendbuff;
        socklen_t optlen;
        int res = 0;
        int i = 0;
    
        /* Boundary values used to test our hypothesis */
            int val[] = {INT_MIN , INT_MIN + 100, INT_MIN + 200, -200 , 0 , 200 , INT_MAX - 200, INT_MAX - 100, INT_MAX};
    
        sockfd = socket(AF_INET, SOCK_DGRAM, 0);
    
        if(sockfd == -1) {
             printf("Error: %s", strerror(errno));
        }
    
        for (i = 0 ; i < 7; i++ ) {
    
            sendbuff = (val[i] / 2.0);
    
            printf("== Setting the send buffer to %d\n", sendbuff);
    
            if (setsockopt(sockfd, SOL_SOCKET, SO_SNDBUFFORCE, &sendbuff, sizeof(sendbuff)) == -1) {
              printf("SETSOCKOPT ERROR: %s\n", strerror(errno));
            }
    
            if (getsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, &sendbuff, &optlen) == -1) {
              printf("GETSOCKOPT ERROR: %s\n", strerror(errno));
            }
            else {
              printf("getsockopt returns buffer size: %d\n", sendbuff);
            }
        }
    
        printf("DONE\n");
    
     return 0;
    }
    
    

    Compile the reproducer:

    [user@target /tmp/]# gcc setsockopt-integer-overflow-ver-001.c -o setsockopt-reproducer
    

    And set the capability of CAP_NET_ADMIN on the binary:

    [user@target /tmp/]# setcap CAP_NET_ADMIN+ep setsockopt-reproducer
    

    If there are exploit creators (or flaw reporters) in the audience, understand that naming your files as reproducer.c and reproducer.py ends up getting confusing, please attempt to create a unique name for files. This can save time when searching through the 200 reproducer.c laying around the file system.

    Saving time

    Virtual machines afford programmers the ability to save the system state for immediate restore. This allows the system to return to a "known good state" if it was to panic or become corrupted. Libvirt calls this kind of snapshot a "System Checkpoint" style snapshot.

    The virt-manager GUI tool in Red Hat Enterprise Linux 7 did not support creating system checkpoints in the GUI. The command line interface is able to create system-checkpoint snapshots by:

    # virsh snapshot-create-as RHEL-7.2-SERVER snapshot-name-1
    

    To restore the system to the snapshot run run the command:

    # virsh snapshot-revert RHEL-7.2-SERVER snapshot-name-1
    

    If the system is running Fedora 20 or newer, and you prefer to use GUI tools, Cole Robinson has written an article which shows how to create system checkpoint style snapshots from within the virt-manager.

    The advantage of snapshots is that you can restore your system back to a working state in case of file system corruption, which can otherwise force you to reinstall from scratch.

    Debugging and inspecting

    GDB contains a "Text User Interface" mode which allows for greater insights into the running code. Start GDB in the "Text User Interface Mode" and connect to the running qemu/kernel using gdb as shown below:

    gdb -tui ~/kernel-debug/var/lib/kernel-3.10.0-327.el7/boot/vmlinux
    
    <gdb prelude here>
    
    (gdb) dir ./usr/src/debug/kernel-3.10.0-327.el7/linux-3.10.0-327.el7.x86_64/
    (gdb) set architecture i386:x86-64:intel
    (gdb) set disassembly-flavor intel
    (gdb) target remote localhost:1234
    
    

    The extra line beginning with dir points GDB to the location of the source used in creating the binary. This allows GDB to show the current line of execution. This directory tree was created when extracting the kernel-debuginfo-package using rpm2cpio.

    GDB should appear similar to the below screenshot:

    The TUI mode will show the source code at the top and the command line interactive session at the bottom window. The TUI can be customized further and this is left as an exercise to the reader.

    Inspecting the value

    The plan was to inspect the value at the time of writing to the sk->sk_sndbuf to determine how different parameters would affect the final value.

    We will set a breakpoint in GDB to stop at that position and print out the value of sk->sk_sndbuf.

    set_sndbuf:
            sk->sk_userlocks |= SOCK_SNDBUF_LOCK;
    >>>>>>>>sk->sk_sndbuf = max_t(u32, val * 2, SOCK_MIN_SNDBUF);  
            /* Wake up sending tasks if we upped the value.  */
            sk->sk_write_space(sk);
            break;
    
        case SO_SNDBUFFORCE:
            if (!capable(CAP_NET_ADMIN)) {
                ret = -EPERM;
                break;
            }
            goto set_sndbuf;
    
    

    The line which assigns the sk->snd_buf value is line 704 in net/core/sock.c. To set a breakpoint on this line we can issue the "break" command to gdb with the parameters of where it should break.

    Additional commands have been appended that will run every time the breakpoint has been hit. In this demonstration the breakpoint will print the value of sk->sk_sndbuf and resume running.

    If you are not seeing the (gdb) prompt, hit ctrl + c to interrupt the system; pausing the system. While the system is suspended in gdb mode it will not take keyboard input or continue any processing.

    (gdb) break net/core/sock.c:703
    Breakpoint 1 at 0xffffffff81516ede: file net/core/sock.c, line 703.
    (gdb) commands
    Type commands for breakpoint(s) 4, one per line.
    End with a line saying just "end".
    >p sk->sk_sndbuf
    >cont
    >end
    (gdb) 
    

    The "command" directive is similar to a function that will be run each time the most recently set breakpoint is run. The 'continue' directive at the (gdb) prompt to resume processing on the target system.

    The plan was to show a binary compare of val to inspect the comparison, however this value was optimized out. GCC would allow us to inspect the 'val' directly if we were to step through the assembly and inspect the registers at the time of comparison. Doing so, however, is beyond the scope of this document.

    Lets give it a simple test running the reproducer against the code with a predictable, commonly used value. Start another terminal, connect to the target node and run the command:

    [user@target]# ./setsockopt-reproducer 
    Setting the send buffer to -1073741824
    getsockopt buffer size: -2147483648
    Setting the send buffer to -1073741774
    getsockopt buffer size: -2147483548
    Setting the send buffer to -1073741724
    getsockopt buffer size: -2147483448
    Setting the send buffer to -100
    getsockopt buffer size: -200
    Setting the send buffer to 0
    getsockopt buffer size: 4608
    Setting the send buffer to 100
    getsockopt buffer size: 4608
    Setting the send buffer to 1073741723
    getsockopt buffer size: 2147483446
    Setting the send buffer to 1073741773
    getsockopt buffer size: 2147483546
    

    At this time there should be a breakpoint showing as executed in the gdb terminal printing out the value every time the function passes net/core/sock.c line 704.

    Breakpoint 4, sock_setsockopt (sock=sock@entry=0xffff88003c57b680, level=level@entry=1, optname=optname@entry=32, optval=<optimized out>, optval@entry=0x7ffce597f1a0 "",
        optlen=optlen@entry=4) at net/core/sock.c:704
    $9 = 212992
    

    The above example shows $? = ______ as the output of the command that we have created. Each dollar ($N) shown in output correspond to the values iterated through in the test-case code.

    int val[] = {INT_MIN , INT_MIN + 1, -1 , 0 , 1 , INT_MAX - 1, INT_MAX};
    

    Listed below is the complete output of the example script:

    Breakpoint 1, sock_setsockopt (sock=sock@entry=0xffff8800366cf680, level=level@entry=1, optname=optname@entry=32, optval=<optimized out>, optval@entry=0x7ffe3ea373c4 "",
        optlen=optlen@entry=4) at net/core/sock.c:704
    $1 = 212992
    
    Breakpoint 1, sock_setsockopt (sock=sock@entry=0xffff8800366cf680, level=level@entry=1, optname=optname@entry=32, optval=<optimized out>, optval@entry=0x7ffe3ea373c4 "2",
        optlen=optlen@entry=4) at net/core/sock.c:704
    $2 = -2147483648
    
    Breakpoint 1, sock_setsockopt (sock=sock@entry=0xffff8800366cf680, level=level@entry=1, optname=optname@entry=32, optval=<optimized out>, optval@entry=0x7ffe3ea373c4 "d",
        optlen=optlen@entry=4) at net/core/sock.c:704
    $3 = -2147483548
    
    Breakpoint 1, sock_setsockopt (sock=sock@entry=0xffff8800366cf680, level=level@entry=1, optname=optname@entry=32, optval=<optimized out>,
        optval@entry=0x7ffe3ea373c4 "\234\377\377\377\003", optlen=optlen@entry=4) at net/core/sock.c:704
    $4 = -2147483448
    
    Breakpoint 1, sock_setsockopt (sock=sock@entry=0xffff8800366cf680, level=level@entry=1, optname=optname@entry=32, optval=<optimized out>, optval@entry=0x7ffe3ea373c4 "",
        optlen=optlen@entry=4) at net/core/sock.c:704
    $5 = -200
    
    Breakpoint 1, sock_setsockopt (sock=sock@entry=0xffff8800366cf680, level=level@entry=1, optname=optname@entry=32, optval=<optimized out>, optval@entry=0x7ffe3ea373c4 "d",
        optlen=optlen@entry=4) at net/core/sock.c:704
    $6 = 4608
    
    Breakpoint 1, sock_setsockopt (sock=sock@entry=0xffff8800366cf680, level=level@entry=1, optname=optname@entry=32, optval=<optimized out>, optval@entry=0x7ffe3ea373c4 "\233\377\377?\003",
        optlen=optlen@entry=4) at net/core/sock.c:704
    $7 = 4608
    
    Breakpoint 1, sock_setsockopt (sock=sock@entry=0xffff8800366cf680, level=level@entry=1, optname=optname@entry=32, optval=<optimized out>, optval@entry=0x7ffe3ea373c4 "\315\377\377?\003",
        optlen=optlen@entry=4) at net/core/sock.c:704
    $8 = 2147483446
    
    
    

    Conclusion

    As we can see, the final values of sk->sk_sndbuf can be below zero if an application manages to set the value incorrectly. There are many areas of the kernel that use sk->sndbuf where the most obvious of places is the tcp_sndbuf_expand function. This value is used and memory is allocated based on this size.

    This is going to be marked as vulnerable in EL7. I leave this as an exercise for the reader to do their own confirmation on other exploits they may be interested in.

    Troubleshooting:

    Listed below are a number of problems that first time users have run into. Please leave problems in the comments and I may edit this article to aid others in finding the solution faster.

    Problem: Can't connect to gdb?
    Solution: Use netstat to check the port is open and listening on the host. Add a rule in the firewall to allow incoming connections to this port.

    Problem: GDB doesn't allow me to type?
    Solution: Hit Ctrl + C to interrupt the current system, enter your command, type 'continue' to resume the hosts execution.

    Problem: Breakpoint is set, but it never gets hit?
    Solution: Its likely that you have a booted kernel and source code mismatch, check to see the running kernel matches the source code/line number that has been set.

    Problem: The ssh connection drops while running the code!
    Solution: If the target system remains in gdbs interrupted mode for too long networked connections to the system can be dropped. Try and connect to the host via "virsh console SOMENAME" to get a non-networked console. You may need to setup a serial console on the host if one is not present.

    Additional thanks to:
    - Doran Moppert (GDB assistance!)
    - Prasad Pandit (Editing)
    - Fabio Olive Leite (Editing)

    Posted: 2017-02-24T14:30:00+00:00
  • Do you know where that open source came from?

    Authored by: Joshua Bressers

    Last year, while speaking at RSA, a reporter asked me about container provenance. This wasn’t the easiest question to answer because there is a lot of nuance around containers and what’s inside them. In response, I asked him if he would eat a sandwich he found on the ground. The look of disgust I got was priceless, but it opened up a great conversation.

    Think about it this way: If there was a ham sandwich on the ground that looked mostly OK, would you eat it? You can clearly see it’s a ham sandwich. The dirt all brushed off. You do prefer wheat bread to white. So what’s stopping you? It was on the ground. Unless you’re incredibly hungry and without any resources, you won’t eat that sandwich. You’ll visit the sandwich shop across the street.

    The other side of this story is just as important though. If you are starving and without money, you’d eat that sandwich without a second thought. I certainly would. Starving to death is far worse than eating a sandwich of questionable origin. This is an example you have to remember in the context of your projects and infrastructure. If you have a team that is starving for time, they aren’t worried about where they get their solutions. For many, making the deadline is far more important than “doing it right.” They will eat the sandwich they find.

    This year at RSA, I’m leading a Peer2Peer session titled, “Managing your open source.” I keep telling everyone that open source won. It’s used everywhere; there’s no way to escape it anymore. But a low-cost, flexible, and more secure software option must have some kind of hidden downside, right? Is the promise of open source too good to be true? Only if you don’t understand the open source supply chain.

    Open source is everywhere, and that means it’s easily acquirable. From cloning off of github to copying random open source binaries downloaded from a project page, there’s no stopping this sort of behavior. If you try, you will fail. Open source won because it solves real problems and it snuck in the back door when nobody was looking. It’s no secret how useful open source is: by my guesstimates, the world has probably saved trillions in man hours and actual money thanks to all the projects that can be reused. If you try to stop it now it’s either going to go back underground, making the problem of managing your open source usage worse or, worse still, you’re going to have a revolt. Open source is amazing, but there is a price for all this awesome.

    Fundamentally, this is our challenge: How do we empower our teams to make the right choices when choosing open source software?

    We know they’re going to use it. We can’t control every aspect of its use, but we can influence its direction. Anyone who is sensitive to technical debt will understand that open source isn’t a “copy once and forget” solution. It takes care and attention to ensure that you haven’t just re-added Heartbleed to your infrastructure. Corporate IT teams need to learn how to be the sandwich shop - how do we ensure that everyone is coming to us for advice and help with open source instead of running whatever they find on the ground? There aren’t easy answers to all of these questions, but we can at least start the discussion.

    In my RSA Peer2Peer session we’re going to discuss what this all means in the modern enterprise:
    - How are you managing your open source?
    - Are you doing nothing?
    - Do you have a program where you vet the open source used to ensure a certain level of quality?
    - How do you determine quality?
    - Are you running a scanner that looks for security flaws?
    - What about the containers or Linux distribution you use, where did that come from, who is taking care of it?
    - How are you installing your open source applications on your Linux or even Windows servers?

    There are a lot of questions, too many to ask in a single hour or day, and far too many to effectively answer over the course of a career in IT security. That’s okay though; we want to start a discussion that I expect will never end.

    See you at RSA on Tuesday February 14, 2017 | 3:45 PM - 4:30 PM | Marriott Marquis | Nob Hill C

    Posted: 2017-02-08T14:30:00+00:00
  • Debugging a kernel in QEMU/libvirt

    Authored by: Wade Mealing

    A kernel bug announced on oss-security list claims to create a situation in which memory corruption can panic the system, by causing an integer used in determining the size of TCP send and receive buffers to be a negative value. Red Hat engineering sometimes backports security fixes and features from the current kernel, diverging the Red Hat Enterprise Linux kernel from upstream and causing some security issues to no longer apply. This blog post shows how to use live kernel debugging to determine if a system is at risk by this integer overflow flaw.

    This walkthrough assumes that the reader has a Red Hat Enterprise Linux 7 guest system and basic knowledge of C programming.

    Setting up the guest target to debug

    The guest to be the target of the debugging session is a libvirt (or KVM/QEMU) style Virtual Machine. The guest virtual serial port should be mapped to the TCP port (TCP/1234) for use by GDB (the GNU Debugger).

    Modifying the guest domain file

    The virsh-edit command is intended to be a safe method of manipulating the raw XML that describes the guest, which is what we need to do in this circumstance. We need to configure the guest via the domain configuration file as there is no tickbox to enable what we need in virt-manager.

    The first change is to set the XML namespace for QEMU, which sounds more complex than it is.

    # virsh-edit your-virt-host-name
    

    Find the domain directive and add the option xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'.

    <domain type='kvm'
            xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' >
    

    Add a new qemu:commandline tag inside domain which will allow us to pass a parameter to QEMU for this guest when starting.

    <domain type='kvm'
           xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' >
         <qemu:commandline>
              <qemu:arg value='-s'/>
         </qemu:commandline>
    

    Save the file and exit the editor. Some versions of libvirt may complain that the XML has invalid attributes; ignore this and save the file anyway. The libvirtd daemon does not need to be restarted. The guest will need to be destroyed and restarted if it is already running.

    The -s parameter is an abbreviation of -gdb tcp::1234. If you have many guests needing debugging on different ports, or already have a service running on port 1234 on the host, you can set the port in the domain XML file as shown below:

      <qemu:commandline>
            <qemu:arg value='-gdb'/>
            <qemu:arg value='tcp::1235'/>
      </qemu:commandline>
    

    If it is working, the QEMU process on the host will be listening on the port specified as shown below:

    [root@target]# netstat -tapn | grep 1234
    tcp        0      0 0.0.0.0:1234            0.0.0.0:*               LISTEN      11950/qemu-system-x
    
    Change /etc/default/grub on the guest

    The guest kernel will need to be booted with new parameters to enable KGDB debugging facilities. Add the values kgdboc=ttyS0,115200. In the system shown here, a serial console is also running on ttyS0 with no adverse effects.

    Use the helpful grubby utility to apply these changes across all kernels.

    # grubby --update-kernel=ALL --args="console=ttyS0,115200 kgdboc=ttyS0,115200"
    
    Downloading debuginfo packages

    The Red Hat Enterprise Linux kernel packages do not include debug symbols, as the symbols are stripped from binary files at build time. GDB needs those debug symbols to assist programmers when debugging. For more information on debuginfo see this segment of the Red Hat Enteprise Linux 7 developer guide.

    RPM packages containing the name 'debuginfo' contain files with symbols. These packages can be downloaded from Red Hat using yum or up2date.

    To download these packages on the guest:

    # debuginfo-install --downloadonly kernel-3.10.0-327.el7
    

    This should download two files in the current directory on the host for later extraction and use by GDB.

    Copy these files from the guest to the host for the host GDB to use them. I choose ~/kernel-debug/ as a sane location for these files. Create the directory if it doesn't already exist.

    # mkdir -p ~/kernel-debug
    # scp yourlogin@guest:kernel*.rpm ~/kernel-debug/
    

    The final step on the guest is to reboot the target. At this point the system should reboot with no change in behavior.

    Preparing the host to debug

    The system which runs the debugger doesn't need to be the host that contains the guest. The debugger system must be capable of making a connection to the guest running on the specified port (1234). In this example these commands will be run on the host which contains the virtual machine.

    Installing GDB

    Install GDB on the host using a package manager.

    # sudo yum -y install gdb
    
    Extracting files to be used from RPMs

    When Red Hat builds the kernel it strips debugging symbols from the RPMs. This creates smaller downloads and uses less memory when running. The stripped packages are the well-known RPM packages named like kernel-3.10.0-327.el7.x86_64.rpm. The non-stripped debug information is stored in debuginfo rpms, like the ones downloaded earlier in this document by using debuginfo-install. They must match the exact kernel version and architecture being debugged on the guest to be of any use.

    The target does not need to match the host system architecture or release version. The example below can extract files from RPMs on any system.

    # cd ~/kernel-debug
    # rpm2cpio kernel-debuginfo-3.10.0-327.el7.x86_64.rpm | cpio -idmv
    # rpm2cpio kernel-debuginfo-common-3.10.0-327.el7.x86_64.rpm | cpio -idmv
    

    This extracts the files within the packages into the current working directory as they would be on the intended file system. No scripts or commands within the RPMs are ran. These files are not installed and the system package management tools will not manage them. This allows them to be used on other architectures, releases or distributions.

    • The unstripped kernel is the vmlinux file in ~/kernel-debug/usr/lib/debug/lib/modules/3.10.0-510.el7.x86_64/vmlinux
    • The kernel source is in the directory ~/kernel-debug/usr/src/debug/kernel-3.10.0-327.el7/linux-3.10.0-327.el7.x86_64/
    Connecting to the target system from the remote system

    Start GDB with the text user interface, passing as a parameter the path to the unstripped kernel binary (vmlinux) running on the target system.

    # gdb -tui ~/kernel-debug/var/lib/kernel-3.10.0-327.el7/boot/vmlinux
    
    <gdb prelude shows here>
    

    GDB must be told where to find the target system. Type the following into the GDB session:

    set architecture i386:x86-64:intel
    target remote localhost:1234
    dir ~/kernel-debug/usr/src/debug/kernel-3.10.0-327.el7/linux-3.10.0-327.el7.x86_64/
    

    Commands entered at the (gdb) prompt can be saved in ~/.gdbinit to reduce repetitive entry.

    At this point, if all goes well, the system should be connected to the remote GDB session.

    The story so far...

    Congratulations, you've made it this far. If you've been following along you should have setup a GDB session to a system running in libvirt and be able to recreate and begin investigation into flaws.

    Join the dark side as next time when we validate an integer promotion comparison flaw.

    Posted: 2017-01-11T14:30:00+00:00
  • Deprecation of Insecure Algorithms and Protocols in RHEL 6.9

    Authored by: Nikos Mavrogian...

    Cryptographic protocols and algorithms have a limited lifetime—much like everything else in technology. Algorithms that provide cryptographic hashes and encryption as well as cryptographic protocols have a lifetime after which they are considered either too risky to use or plain insecure. In this post, we will describe the changes planned for the 6.9 release of Red Hat Enterprise Linux 6, which is already on Production Phase 2.

    Balancing Legacy Use Cases and Modern Threats

    For the RHEL operating system, which has its useful lifetime measured in decades, in addition to adding new features, it is necessary to periodically revisit default settings and phase out (or completely deprecate) protocols and algorithms the use of which—accidental or intentional—could cause irreparable damage. At the same time we must ensure that the vast majority of existing and legacy applications continues to operate without changes, as well as provide mechanisms for the administrator to revert any changes, when necessary, to the previous defaults.

    What Are the Threats?

    Given that any change in application or library default settings cannot be without side-effects, it is important to provide the context under which such changes are necessary. In the past few years, we’ve identified several protocol attacks with real-world impact that relied on obsolete and legacy algorithm and protocol configurations. A few prominent attacks are briefly described below:

    • DROWN in 2016; it relied on servers enabling the SSL 2.0 protocol, allowing the attackers to obtain sufficient information to decrypt other, unrelated TLS sessions.
    • SLOTH in 2016; it relied on clients enabling the MD5 algorithm, broken since 2004; it allowed attackers to decrypt TLS sessions.
    • LOGJAM and FREAK in 2015. While the details of the attacks differ, both of these attacks relied on export cryptography being enabled in the server, allowing an attacker to decrypt TLS sessions.

    Why Would Insecure Configuration Remain Enabled?

    While all of the exploited protocols and algorithms were known to be obsolete or insecure for more than a decade, the impact of these attacks was still high. That indicates that these obsolete protocols and algorithms were enabled on Internet servers possibly due to:

    • misconfiguration,
    • administrators’ hope of improving compatibility with legacy clients,
    • re-use of old configuration files.

    Traditionally, we have not been very keen on deprecating algorithms and protocols throughout the RHEL timeline to avoid breaking existing and legacy applications. This was because of our belief that for an operating system, keeping operations going on is more important than addressing flaws that may not be applicable on every operating system setup.

    Solution for RHEL 6.9

    However, after considering these attacks and the fact that it is unrealistic to expect all administrators to keep up with cryptographic advances, we have decided to provide a protection net, which will prevent future cryptographic attacks due to accidental misconfiguration with legacy algorithms and protocols in default RHEL 6.9 installations.

    No Export Ciphersuites and SSL 2.0

    In particular, we will take steps that will ensure that the TLS ciphersuites marked as export, as well as the SSL 2.0 protocol, are completely removed from the operating system. These two points involve algorithms with no real-world use and a protocol that has been considered deprecated for more than 20 years. We will not provide a way to re-enable these options because we are convinced that these are primarily used to attack misconfigured systems rather than for real-world use cases.

    Limited MD5, RC4, and DH

    In addition, we will ensure that no application can be used with insecure signature-verification algorithms such as MD5, and that TLS client applications refuse to communicate with servers that advertise less than 1024-bit Diffie-Hellman parameters. The latter would ensure that LOGJAM-style of attacks are prevented. Furthermore, we will disable the usage of RC4 in contexts where this will not introduce compatibility issues.

    All-around Application Support for TLS 1.2

    While deprecating insecure algorithms and protocols protects applications running in RHEL from future attacks taking advantage of them, it is also important, given that RHEL 6.9 entered Production Phase 2, to provide a solid cryptographic base for the remaining lifetime of the product. For that we will ensure that all the back-end cryptographic components support TLS 1.2, allowing new applications to be deployed and used during its lifetime.

    Summary of Changes to Cryptographic Back-ends in RHEL 6.9

    Introduced change Description Revertable
    TLS 1.2 protocol The protocol is made available to all shipped cryptographic libraries and enabled by default. N/A
    SSL 2.0 protocol The shipped TLS libraries will no longer include support for the SSL 2.0 protocol. No
    Export TLS ciphersuites The shipped TLS libraries will no longer include support for Export TLS ciphersuites. No
    TLS Diffie-Hellman key exchange Only parameters larger than 1024-bits will be accepted by default.1 Yes. Administrators can revert this setting system-wide (information will provided in the release notes).
    MD5 algorithm for digital signatures The algorithm will not be enabled by default for TLS sessions or certificate verification on any of the TLS libraries we ship. Yes. Administrators can revert this setting system-wide (information will provided in the release notes).
    RC4 algorithm The algorithm will no longer be enabled by default for OpenSSH sessions. Yes. Administrators can revert this setting system-wide (information will provided in the release notes).

    1. This, exceptionally, applies only to OpenSSL, GnuTLS, and NSS cryptographic back-ends, not to Java/OpenJDK. 

    Posted: 2017-01-03T14:30:00+00:00
  • Pythonic code review

    Authored by: Ilya Etingof

    Most of us programmers go through technical interviews every once in a while. At other times, many of us sit on the opposite side of the table running these interviews. Stakes are high, emotions run strong, intellectual pressure builds up. I have found that an unfortunate code review may turn into something similar to a harsh job interview.

    While it is theoretically in the best interest of the whole team to end up with high quality code, variations in individual's technical background, cultural differences, preconceptions built up on previous experience, personality quirks, and even temper may lure people into a fierce fight over relatively unimportant matters.

    Consider an imaginary pull request. There we typically have two actors: the author and code reviewers. Sometimes authors tend to overestimate the quality of their code which provokes them to be overly defensive and possibly even hostile to any argument. People reviewing the code may find themselves being in a position of power to judge author's work. Once the actors collide over a matter where they take orthogonal and sufficiently strong sides, all is fair in love and war.

    Another interesting phenomena I encountered while reviewing Python code can probably be attributed to Python's low barrier to entry for newcomers. Programmers switching over from other languages bring along customs and idioms they are used to in their "mother tongue". I can usually figure out from their Python code whether the author is a former Java, Perl, or Bash programmer. As much as I admire other technologies and expertise, I believe it is most efficient and enjoyable to code in harmony with the language rather than stretching it beyond its intended design.

    In this article I'll focus on my personal experience in authoring and reviewing Python code from both psychological and technical perspectives. And I'll do so keeping in mind the ultimate goal of striking a balance between code reviews being enjoyable and technically fruitful.

    Why we review code

    The most immediate purpose of a code review is to make code better by focusing more eyeballs on it. But code review seems to be a lot more than that!

    It is also a way for the engineers to communicate, learn and even socialize over a meaningful and mutually interesting topic. In a team where both senior and junior engineers work together, a code review provides the opportunity for the junior engineers to observe masters at work and learn from them.

    Seniors, in turn, get a chance to coach fellow engineers, be challenged and thus prove their authority (which is healthy). Everyone can see the problem from different perspectives, which ultimately contributes to a better outcome.

    For mature Pythonistas coding with newbies, code review is a way to teach them how we do things in Python with the goal of creating idiomatic Python coders out of them.

    For the greater good

    The best we can do on the psychological side is to, as an author, relinquish our emotional attachment to our code and, as a reviewer, consciously restrain ourselves from attacking other authors' ideas, focusing instead on mentoring them.

    For a review to be a productive and relatively comfortable experience, a reviewer should stay positive, thankful, honestly praise the author's work and talent in a genuine matter. Suggested changes should be justified by solid technical grounds, and never by the reviewer's personal taste.

    For authors it may help to keep reminding themselves how much time and effort it might have taken for reviewers to work with the author's code -- their feedback is precious!

    As idealistic as it sounds, my approach aims at downplaying my ego by optimizing for a healthier team and an enjoyable job. I suspect that it may come at the cost of compromised quality of the code we collectively produce, but to me that is worth it. My hope here is that even if we do merge sub-optimal code at times, we will eventually learn from that and re-factor later. That's a much cheaper cost compared to ending up with a stressed, despaired and demotivated colleague.

    When I'm an author

    As an author, I'm not taking Pull Requests (PRs) lightly. My day's worth of code is likely to keep fellow reviewers busy for a good couple of hours. I know that it's a hard and expensive endeavor. Proper code review may require my team mates to reverse engineer business logic behind a change, trace code execution, conduct thought experiments, and search for edge cases. I do keep that in mind and feel grateful for their time.

    I try to keep my changes small. The larger the change is, the more effort it would take for reviewers to finish the review. That gives smaller, isolated changes a better chance for quality treatment, while huge and messy blobs of diffs risk turning a blind eye on my PR.

    Well-debugged code accompanied with tests, properly documented changes complying with team policies -- those PR qualities are signs of respect and care towards reviewers. I feel myself more confident once I run a quick self code review against my prospective PR prior to submitting it to fellow engineers.

    When I'm a reviewer

    To me, the most important qualities of code is to be clean and as Pythonic as circumstances permit.

    Clean code tends to be well-structured, where logically distinct parts are isolated from each other via clearly visible boundaries. Ideally, each part is specialized on solving a single problem. As Zen of Python puts it: "If the implementation is easy to explain, it may be a good idea".

    Signs of clear code include self-documented functions and variables describing problem entities, not implementation details.

    Readability counts, indeed, though I would not sweat full PEP8 compliance where it becomes nitpicking and bikesheding.

    I praise the author's coding on the shoulders of giants -- abstracting problems into canonical data structures/algorithms and working from there. That gives me the warm feeling of the author belonging to the same trade guild as myself, and confidence that we both know what to expect from the code.

    When I'm a Pythonista

    The definition of code being Pythonic tends to be somewhat vague and subjective. It seems to be influenced by one's habits, taste, picked up idioms and language evolution. I keep that in mind and restrain myself from evangelizing my personal perception of what's Pythonic towards fellow Pythonistas.

    Speaking from my experience in the field, let me offer the reader a handful of code observations I have encountered, along with suggestions leveraging features that are native to Python of which people with different backgrounds may be unaware.

    Justified programming model

    People coming from Java tend to turn everything into a class. That's probably because Java heavily enforces the OOP paradigm. Python programmers enjoy a freedom of picking a programming model that is best suited for the task.

    The choice of object-based implementations look reasonable to me when there is a clear abstraction for the task being solved. Statefulness and duck-typed objects are another strong reason for going the OOP way.

    If the author's priority is to keep related functions together, pushing them to a class is an option to consider. Such classes may never need instantiation, though.

    Free-standing functions are easy to grasp, concise and light. When a function does not cause side-effects, it's also good for functional programming.

    Pythonic loops

    Folks coming from C might feel at home with index-based while loops:

        # Non-Pythonic
        choices = ['red', 'green', 'blue']
        i = 0
        while i < len(choices):
            print('{}) {}'.format(i, choices[i]))
            i += 1
    

    or with for loops like this:

        # Non-Pythonic
        choices = ['red', 'green', 'blue']
        for i in range(len(choices)):
            print('{}) {}'.format(i, choices[i]))
    

    When the index is not required, a more Pythonic way would be to run a for loop over a collection:

        # Pythonic!
        choices = ['red', 'green', 'blue']
        for choice in choices:
            print(choice)
    

    Otherwise enumerate the collection and run a for loop over the enumeration:

        # Pythonic!
        choices = ['red', 'green', 'blue']
        for idx, choice in enumerate(choices):
            print('{}) {}'.format(idx, choice))
    

    As a side note, Python's for loop is quite different from what we have in C, Java or JavaScript. Technically, it's a foreach loop.

    What if we have many collections to loop over? As naive as it could get:

        # Non-Pythonic
        preferences = ['first', 'second', 'third']
        choices = ['red', 'green', 'blue']
        for i in range(len(choices)):
            print('{}) {}'.format(preferences[i], choices[i]))
    

    But there is a better way -- use zip:

        # Pythonic!
        preferences = ['first', 'second', 'third']
        choices = ['red', 'green', 'blue']
        for preference, choice in zip(preferences, choices):
            print('{}) {}'.format(preference, choice))
    

    Comprehensions, no tiny loops

    Even a perfectly Pythonic loop can be further improved by turning it into a list or dictionary comprehension. Consider quite a mundane for-loop building a sub-list on a condition:

        # Non-Pythonic
        oyster_months = []
        for month in months:
            if 'r' in month:
                oyster_months.append(month)
    

    List comprehension reduces the whole loop into a one-liner!

        # Pythonic!
        oyster_months = [month for month in months if 'r' in month]
    

    Dictionary comprehension works similarly, but for mapping types.

    Readable signatures

    Differing from many languages, in Python, names of function parameters are always part of function signature:

        >>> def count_fruits(apples, oranges):
        ...     return apples + oranges
        ... 
        >>> count_fruits(apples=12, oranges=21)
        >>> count_fruits(garlic=14, carrots=12)
        TypeError: count_fruits() got an unexpected keyword argument 'garlic'
    

    The outcome is twofold: a caller can explicitly refer to a parameter name to improve code readability. The function's author should be aware of callers possibly binding to a once-announced name and restrain from changing names in public APIs.

    At any rate, passing named parameters to the functions we call adds to code readability.

    Named tuples for readability

    Wrapping structured stuff into a tuple is a recipe for communicating multiple items with a function. Trouble is that it quickly becomes messy:

        # Non-Pythonic
        >>> team = ('Jan', 'Viliam', 'Ilya')
        >>> team
        ('Jan', 'Viliam', 'Ilya')
        >>> lead = team[0]
    

    Named tuples simply add names to tuple elements so that we can enjoy object notation for getting hold of them:

        # Pythonic!
        >>> Team = collections.namedtuple('Team', ['lead', 'eng_1', 'eng_2'])
        >>> team = Team('Jan', 'Viliam', 'Ilya')
        >>> team
        Team(lead='Jan', eng_1='Viliam', eng_2='Ilya')
        >>> lead = team.lead
    

    Using named tuples improves readability at the cost of creating an extra class. Keep in mind, though, that namedtuple factory functions create a new class by exec'ing a template -- that may slow things down in a tight loop.

    Exceptions, no checks

    Raising an exception is a primary vehicle for communicating errors in a Python program. It's easier to ask for forgiveness than permission, right?

        # Non-Pythonic
        if resource_exists():
            use_resource()
    
        # Pythonic!
        try:
            use_resource()
        except ResourceDoesNotExist:
            ...
    

    Beware likely failing exceptions in tight loops, though -- those may slow down your code.

    It is generally advisable to subclass built-in exception classes. That helps to clearly communicate errors that are specific to our problem and differentiate errors that bubble up to our code from other, less expected failures.

    Ad-hoc namespaces

    When we encounter colliding variables, we might want to isolate them from each other. The most obvious way is to wrap at least one of them into a class:

        # Non-Pythonic
        class NS:
            pass
    
        ns = NS()
        ns.fruits = ['apple', 'orange']
    

    But there is a handy Pythonic shortcut:

        # Pythonic!
        ns = types.SimpleNamespace(fruits=['apple', 'orange'])
    

    The SimpleNamespace object acts like any class instance -- we can add, change or remove attributes at any moment.

    Dictionary goodies

    Python dict is a well-understood canonical data type much like a Perl hash or a Java HashMap. In Python, however, we have a few more built-in features like returning a value for a missing key:

        >>> dict().get('missing key', 'failover value')
        'failover value'
    

    Conditionally setting a key if it's not present:

        >>> dict().setdefault('key', 'new value')
        'new value'
        >>> d = {'key': 'old value'}
        >>> d.setdefault('key', 'new value')
        'old value'
    

    Or automatically generate an initial value for missing keys:

        >>> d = collections.defaultdict(int)
        >>> d['missing key'] += 1
        >>> d['missing key']
        1
    

    A dictionary that maintains key insertion order:

        >>> d = collections.OrderedDict()
        >>> d['x'] = 1
        >>> d['y'] = 1
        >>> list(d)
        ['x', 'y']
        >>> del d['x']
        >>> d['x'] = 1
        >>> list(d)
        ['y', 'x']
    

    Newcomers may not be aware of these nifty little tools -- let's tell them!

    Go for iterables

    When it comes to collections, especially large or expensive ones to compute, the concept of iterability kicks right in. To start with, a for loop implicitly operates over Iterable objects:

        for x in [1, 2, 3]:
            print(x)
    
        for line in open('myfile.txt', 'r'):
            print(line)
    

    Many built-in types are already iterable. User objects can become iterable by supporting the iterator protocol:

        class Team:
            def __init__(self, *members):
                self.members = members
                self.index = 0
    
            def __iter__(self):
                return self
    
            def __next__(self):
                try:
                    return self.members[self.index]
                except IndexError:
                    raise StopIteration
                finally:
                    self.index += 1
    

    so they can be iterated over a loop:

        >>> team = Team('Jan', 'Viliam', 'Ilya')
        >>> for member in team:
        ...     print(member)
        ...         
        Jan
        Viliam
        Ilya
    

    as well as in many other contexts where an iterable is expected:

        >>> team = Team('Jan', 'Viliam', 'Ilya')
        >>> reversed(team)
        ['Ilya', 'Viliam', 'Jan']
    

    Iterable user functions are known as generators:

        def team(*members):
            for member in members:
                yield member
    
        >>> for member in team('Jan', 'Viliam', 'Ilya'):
        ...     print(member)
        ...     
        ... 
        Jan
        Viliam
        Ilya
    

    The concept of an iterable type is firmly built into the Python infrastructure and it is considered Pythonic to leverage iterability features.

    Besides being handled by built-in operators, there is a collection of functions in the itertools module that are designed to consume and/or produce iterables.

    Properties for gradual encapsulation

    Java and C++ are particularly famous for promoting object state protection by operating via "accessor" methods (also known as getters/setters). A Pythonic alternative to them is based on the property feature. Unlike Java programmers, Pythonistas do not begin with planting getters and setters into their code. They start out with simple, unprotected attributes:

        class Team:
            members = ['Jan', 'Viliam', 'Ilya']
    
        team = Team()
        print(team.members)
    

    Once a need for protection arises, we turn an attribute into a property by adding access controls into the setter:

        class Team:
            _members = ['Jan', 'Viliam', 'Ilya']
    
            @property
            def members(self):
                return list(self._members)
    
            @members.setter
            def members(self, value):
                raise AttributeError('This team is too precious to touch!')
    
        >>> team = Team()
        >>> print(team.members)
        ['Jan', 'Viliam', 'Ilya']
        >>> team.members = []
        AttributeError('This team is too precious to touch!',)
    

    Python properties are implemented on top of descriptors which is a lower-level but more universal mechanism to get hold on attribute access.

    Context managers to guard resources

    It is common for programs to acquire resources, use them, and clean up afterwards. A simplistic implementation might look like this:

        # Non-Pythonic
        resource = allocate()
        try:
            resource.use()
        finally:
            resource.deallocate()
    

    In Python, we could re-factor this into something more succinct, leveraging the context manager protocol:

        # Pythonic!
        with allocate_resource() as resource:
            resource.use()
    

    The expression following a with must support the context manager protocol. Its __enter__ and __exit__ magic methods will be called respectively before and after the statements inside the with block.

    Context managers are idiomatic in Python for all sorts of resource control situations: working with files, connections, locks, processes. To give a few examples, this code will ensure that a connection to a web server is closed once the execution runs out of with block:

        with contextlib.closing(urllib.urlopen('https://redhat.com')) as conn:
            conn.readlines()
    

    The suppress context manager silently ignores the specified exception if it occurs within the body of the with statement:

        with contextlib.suppress(IOError):
            os.unlink('non-existing-file.txt')
    

    Decorators to add functionality

    Python decorators work by wrapping a function with another function. Use cases include memoization, locking, pre/post-processing, access control, timing, and many more.

    Consider a straightforward memoization implementation:

        # Non-Pythonic
        cache = {}
        def compute(arg):
            if arg not in cache:
                cache[arg] = do_heavy_computation(arg)
            return cache[arg]
    

    This is where Python decorators come in handy:

        def universal_memoization_decorator(computing_func):
            cache = {}
            def wrapped(arg):
                if arg not in cache:
                    cache[arg] = computing_func(arg)
                return cache[arg]
            return wrapped
    
        # Pythonic!
        @universal_memoization_decorator
        def compute(arg)
            return do_heavy_computation(arg)
    

    The power of decorators comes from their ability to modify function behavior in a non-invasive and universal way. That opens up possibilities to offload business logic into a specialized decorator and reuse it across the whole codebase.

    The Python standard library offers many ready-made decorators. For example, the above memorization tool is readily available in the standard library:

        # Pythonic!
        @functools.lru_cache()
        def compute(arg):
            return do_heavy_computation(arg)
    

    Decorators have made their way into public APIs in large projects like Django or PyTest.

    Duck typing

    Duck typing is highly encouraged in Python for being more productive and flexible. A frequent use-case involves emulating built-in Python types such as containers:

        # Pythonic!
        class DictLikeType:
            def __init__(self, *args, **kwargs):
                self.store = dict(*args, **kwargs)
    
            def __getitem__(self, key):
                return self.store[key]
    
            ...
    

    Full container protocol emulation requires many magic methods to be present and properly implemented. This can become laborious and error prone. A better way is to base user containers on top of a respective abstract base class:

        # Extra Pythonic!
        class DictLikeType(collections.abc.MutableMapping):
            def __init__(self, *args, **kwargs):
                self.store = dict(*args, **kwargs)
    
            def __getitem__(self, key):
                return self.store[key]
    
            ...
    

    Not only would we have to implement less magic methods, the ABC harness ensures that all mandatory protocol methods are in place. This partly mitigates the inherent fragility of dynamic typing.

    Goose typing

    Type checking based on types hierarchy is a popular pattern in Python programs. People with a background in statically-typed languages tend to introduce ad-hoc type checks like this:

        # Non-Pythonic
        if not isinstance(x, list):
            raise ApplicationError('Python list type is expected')
    

    While not discouraged in Python, type checks could be made more general and reliable by testing against abstract base types:

        # Pythonic!
        if not isinstance(x, collections.abc.MutableSequence):
            raise ApplicationError('A sequence type is expected')
    

    This is known as "Goose typing" in Python parlance. It immediately makes a type check compatible with both built-in and user types that inherit from abstract base classes. Additionally, checking against an ABC empowers interface-based, as opposed to hierarchy-based, types comparison :

        class X:
            def __len__(self):
                return 0
    
        >>> x = X()
        >>> isinstance(x, collections.abc.Sized)
        True
    

    Alternative to ad-hoc type checks planted into the code is the gradual typing technique which is fully supported since Python 3.6. It is based on the idea of annotating important variables with type information, then running a static analyzer over the annotated code like this:

        def filter_by_key(d: typing.Mapping, s: str) -> dict:
            return {k: d[k] for k in d if k == s}
    
        d: dict = filter_by_key('x': 1, 'y': 2}, 'x')
    

    Static typing tends to make programs more reliable by leveraging explicit type information, computing types compatibility and failing gracefully when a type error is discovered. When type hinting is adopted by a project, type annotations can fully replace ad-hoc type checks throughout the code.

    Pythonista's power tools

    It may occur to a reviewer, that a more efficient solution is possibly viable here and there. To establish solid technical grounds by backing their re-factoring proposal with hard numbers rather than intuition or personal preference, a quick analysis may come in handy.

    Among the tools I use when researching for a better solution are dis (for bytecode analysis), timeit (for code snippets running time measurement) and profile (for finding hot spots in a running Python program).

    Happy reviewing!

    Posted: 2016-12-14T14:30:00+00:00
  • Evolution of the SSL and TLS protocols

    Authored by: Huzaifa Sidhpurwala

    The Transport Layer Security (TLS) protocol is undoubtedly the most widely used protocol on the Internet today. If you have ever done an online banking transaction, visited a social networking website, or checked your email, you have most likely used TLS. Apart from wrapping the plain text HTTP protocol with cryptographic goodness, other lower level protocols like SMTP and FTP can also use TLS to ensure that all the data between client and server is inaccessible to attackers in between. This article takes a brief look at the evolution of the protocol and discusses why it was necessary to make changes to it.

    Like any other standard used today on the internet, the TLS protocol also has a humble beginning and a rocking history. Originally developed by Netscape in 1993 it was initially called Secure Sockets Layer (SSL). The first version was said to be so insecure that "it could be broken in ten minutes" when Marc Andreessen presented it at an MIT meeting. Several iterations were made which led to SSL version 2 and, later in 1995, SSL version 3. In 1996, an IETF working group formed to standardize SSL. Even though the resulting protocol is almost identical to SSL version 3, the process took three years.

    TLS version 1.0, with a change in name to prevent trademark issues, was published as RFC 2246. Later versions 1.1 and 1.2 were published which aimed to address several shortcomings and flaws in the earlier versions of the protocol.

    Cryptographic primitives are based on mathematical functions and theories

    The TLS protocol itself is based on several cryptographic primitives including asymmetric key exchange protocols, ciphers, and hashing algorithms. Assembling all these primitives together securely is non-trivial and would not be practical to implement individually in the same way TLS does. For example, AES is a pretty strong symmetric cipher, but like any other symmetric cipher it needs the encryption key to be securely exchanged between the client and the server. Without an asymmetric cipher there is no way to exchange keys on an insecure network such as the Internet. Hashing functions are used to help authenticate the certificates used to exchange the keys and also ensure integrity of data-in-transit. These hash algorithms, like SHA, have one way properties and are reasonably collision resistant. All these cryptographic primitives, arranged in a certain way, make up the TLS protocol as a whole.

    Key Exchanges

    The reason two systems that have never met can communicate securely is due to secure key exchange protocols. Because each system must know the same secret to establish a secure communications path using a symmetric cipher, the use of key exchange systems allow those two systems to establish that secret and securely share it with each other to establish the communications path.

    The Rivest-Shamir-Adleman (RSA) cryptosystem is the most widely used asymmetric key exchange algorithm. This algorithm assumes that factorization of large numbers is difficult, so while the public key (n) is calculated using n = p x q, it is hard for an attacker to factorize n into the corresponding primes p and q, which can be easily used to calculate the private key.

    The Diffie-Hellman key exchange (DHE) uses the discrete log problem and assumes that when given y = g ^ a mod p, it is difficult to solve this equation to extract the private key a. Elliptic-Curve-based Diffie-Hellman key exchange (ECDHE) uses the abstract DH problem, but uses multiplication in elliptic curve groups for its security.

    Symmetric algorithms

    Symmetric algorithms used today like Advanced Encryption Standard (AES) have good confusion and diffusion properties, which mean that the encrypted data will be statistically different from the input. ChaCha20 is a newer stream cipher that is starting to see some traction and may see additional use in the future as a faster alternative to AES.

    Changes as time and technology progresses

    Faster computers are now more accessible to the common public via cloud computing, GPUs, and dedicated FPGA devices than they were 10 years ago. New computation methods have also become possible. Quantum computers are getting bigger, making possible attacks on the underlying mathematics of many algorithms used for cryptography. Also, new research in mathematics means that as older theories are challenged and newer methods are invented and researched, our previous assumptions about hard mathematical problems are losing ground.

    New design flaws in the TLS protocol are also discovered from time to time. The POODLE flaw in SSL version 3 and DROWN flaw in SSL version 2 showed that the previous versions of the protocol are not secure. We can likely expect currently deployed versions of TLS to also have weaknesses as research continues and computing power gets greater.

    Attacks against cryptographic primitives and its future

    RSA

    The best known attack against RSA is still factoring n into its components p and q. The best known algorithm for factoring integers larger than 10^100 is the number field sieve. The current recommendation from NIST is using a minimum RSA key length of 2048 bits for information needed to be protected until at least the year 2030. For secrecy beyond that year larger keys will be necessary.

    RSA's future, however, is bleak! IETF recommended removal of static-RSA from the TLS version 1.3 draft standard stating "[t]hese cipher suites have several drawbacks including lack of PFS, pre-master secret contributed only by the client, and the general weakening of RSA over time. It would make the security analysis simpler to remove this option from TLS version 1.3. RSA certificates would still be allowed, but the key establishment would be via DHE or ECDHE." The consensus in the room at IETF-89 was to remove RSA key transport from TLS 1.3.

    DHE and ECC

    Like RSA, the best known attack against DHE is the number field sieve. With the current computing power available, a 512-bit DH key takes 10 core-years to break. NIST recommends a key size of 224 bits and 2048-bit group size for any information which needs to be protected till 2030.

    As compared to DHE, ECC has still stood its ground and is being increasingly used in newer software and hardware implementations. Most of the known attacks against ECC work only on special hardware or against buggy implementations. NIST recommends use of at least 224-bit key size for ECC curves.

    However, the biggest threat to all of the above key exchange methods is quantum computing. Once viable quantum computing technology is available, all of the above public key cryptography systems will be broken. NIST recently conducted a workshop on post-quantum cryptography and several alternatives to the above public cryptography schemes were discussed. It is going to be interesting to watch what these discussions lead to, and what new standards are formed.

    Symmetric ciphers and hashes

    All symmetric block ciphers are vulnerable to brute force attacks. The amount of time taken to brute force depends on the size of the key; the bigger the key, the more time and power it takes to brute force. The SWEET32 attack has already shown that small block sizes are bad and has finally laid 3DES to rest. We already know that RC4 is insecure and there have been several attempts to deprecate it.

    The proposed TLS version 1.3 draft has provision for only two symmetric ciphers, namely AES and ChaCha20, and introduces authenticated encryption (AEAD). The only MAC function allowed is Poly1305.

    And in conclusion...

    No one knows for sure what will happen next but history has shown that older algorithms are at risk. That's why it is so important to stay up to date on cryptography technology. Developers should make sure their software supports the latest versions of TLS while deprecating older versions that are broken (or weakened). System owners should regularly test their systems to verify what ciphers and protocols are supported and stay educated on what is current and what the risks are to utilizing old cryptography.

    Posted: 2016-11-16T14:30:00+00:00