[BUG][codegen] Nullable references, nullable primitive types and complex types

Created by: sebastien-rosset

Bug Report Checklist

• Have you provided a full/minimal spec to reproduce the issue?
• Have you validated the input using an OpenAPI validator (example)?
• What's the version of OpenAPI Generator used?
• Have you search for related issues/PRs?
• What's the actual output vs expected output?
• [Optional] Bounty to sponsor the fix (example)

Description

There is inconsistent support for nullable types across language generators. Some language generators (e.g. "go") are lenient and allow null values even if the OAS spec does explicitly list nullable types. Other language generators (e.g. Python) do strict validation and raise an Exception if the server returns a null value that wasn't explicitly allowed in the OAS spec.

To complicate matters, there are multiple ways to specify nullable values in the OAS spec, and there are changes between 3.0 and 3.1:

1. In OAS 3.1, the type property now allows "null" as a type, alone or within a type array. This is the standard way to allow null values in JSON Schema.

type: ['null', 'string']
type: 'null'

2. The "nullable" attribute is marked deprecated in 3.1. There are multiple discussion threads about the issues with the "nullable" attribute.

• Clarify Semantics of nullable in OpenAPI
• Reference objects don't combine well with “nullable
• nullable vs type arrays, JSON Schema compatibility
• For discussion only; support JSON Schema Draft 6 in 3.1

openapi-generator version

4.2.3

OpenAPI declaration file content or url

Suppose an OAS spec defines #/definitions/Address of type: object:

Address:
  type: object

In OAS 3.0, Address can be marked as nullable:

Address:
  type: object
  nullable: true

Then other types can reference "Address", such as an "Order" type that has a "billTo" property, and "billTo" can have a null value:

Order:
  type: object
  properties:
    billTo:
      $ref: "#/definitions/Address"

One issue is that at least one language generator (python-experimental) is ignoring the "nullable" attribute. I.e. in the case of Python, when the server returns a null value for "billTo", the client raises an exception because it did not expect a null value. On the other hand, the "go-experimental" code generator accepts null values, even if "nullable: true" wasn't specified in the OAS spec.

Another way to specify a nullable property is to use the vendor extension "x-nullable". A grep shows "x-nullable" is still used in a few OpenAPITool files. Is this still needed? I think this is a remnant of pre-3.0 OAS, before the "nullable" attribute existed.

In addition, "nullable" brings other issues that are described in Clarify Semantics of nullable in OpenAPI, hence it looks like the OAS TSR is deprecating "nullable" in 3.1.

Another problem with the above YAML is that it's not possible to selectively specify which properties of type "address" are nullable or not. For example, suppose the "Order" type has two properties, "shipTo" and "billTo":

Order:
  type: object
  properties:
    shipTo:
      $ref: "#/definitions/Address"
    billTo:
      $ref: "#/definitions/Address"

Let's assume for the sake of this argument that "shipTo" must be set to a non-null value, and "billTo" can have a null value. Using "nullable" inside "Address" would not work, or at least the same "Address" type could not be reused for both the "shipTo" and "billTo" properties.

Order:
  type: object
  properties:
    shipTo:
      $ref: "#/definitions/Address"
    billTo:
      $ref: "#/definitions/Address"

With the above, both "shipTo" and "billTo" can have a null value, which is not what was intended.

To address this issue, one may be tempted to use the nullable attribute as a sibling of $ref. Instead of specifying the nullable attribute in "Address", "nullable" would be specified in the property, as shown below.

Order:
  type: object
  properties:
    shipTo:
      $ref: "#/definitions/Address"
      nullable: false
    billTo:
      $ref: "#/definitions/Address"
      nullable: true

However, the above is not supported in 3.0, the spec explicitly states all sibling attributes of "$ref" are ignored, and that seems to be ignored by the OpenApiTools code generators as well.

Note: the OAS TSR is considering (but nothing is decided) supporting siblings attributes of "$ref" and the associated merge semantic. However it's unlikely this will apply to "nullable" because "nullable" is deprecated anyway.

One way to workaround the above issue is to define two types, "Address" and "OptionalAddress":

OptionalAddress:
  type: object
  nullable: true
  properties:
    zipCode: 
      type: string
    city: 
      type: string

Address:
  allOf:
   - $ref: "#/definitions/OptionalAddress"
  type: object
  #Setting nullable to false is really a no-op, but makes the intent more explicit.
  nullable: false 

Then the "Order" type can be changed as below:

Order:
  type: object
  properties:
    shipTo:
      $ref: "#/definitions/Address"
    billTo:
      $ref: "#/definitions/OptionalAddress"

However this has two problems: the OAS spec becomes verbose (maybe the OAS ends up specifying XYZ types and OptionalXYZ type for everything), and anyway "nullable: false" is a no-op.

Maybe a bigger issue is that "nullable" is being deprecated in 3.1, so authors should stop using nullable and use other constructs to specify nullable properties.

Besides the "nullable" attribute, another way to specify nullable properties is to use "oneOf":

Order:
  type: object
  properties:
    billTo:
      oneOf:
      - type: 'null'
      - $ref: "#/definitions/Address"
    shipTo:
      $ref: "#/definitions/Address"

This seems to be the standard, future-proof way to specify nullable properties for complex types. Note: type: 'null' is not supported in OAS 3.0.x. It is being introduced in OAS 3.1.

From the perspective of the OpenAPITool code generator, there are a few problems:

1. Support for 'oneOf' is incomplete and sometimes not supported at all in the OpenAPITools language generators.
2. The generated code could be very verbose, and casual SDK users will be surprised by the amount of complexity to access what they thought was just a simple value. In this particular case, it would be nice if the generated code has a simple Get() method that returns a pointer to the instance, and that pointer can be null. I'm oversimplifying, but I'm contrasting this to a fully expanded "oneOf" construct which can be very verbose in the generated code. This may require special case handling in codegen.

It would be really nice if the generated code is simple, and looks the same whether it was written using "oneOf" or nullable. However, this does not seem very simple to achieve because the OAS spec could have more than oneOf types, such as below:

Order:
  type: object
  properties:
    billTo:
      oneOf:
      - type: 'null'
      - $ref: "#/definitions/Address"
      - $ref: "#/definitions/BitcoinUrl"

Finally, for properties that are of a primitive type, and the type should accept a null value, it's possible to use type arrays in OAS 3.1. This is the equivalent of "oneOf" for primitive types:

Address:
  type: object
  properties:
    addressLine1:
      type: ['null', 'string']

which is equivalent to:

Address:
  type: object
  properties:
    addressLine1:
      oneOf:
      - type: 'null'
      - type: 'string'

Command line used for generation

Steps to reproduce

Related issues/PRs

Suggest a fix