designate-specs

Asynchronous Zone Import/Export

Mon, 23 Mar 2026 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/async-import-export

Problem description

Large zone imports have the potential to take a lot longer than the typically allowed time for API responses. Parsing large zone files, and inserting the necessary records scales linearly with the amount of records that need to be imported.

Proposed change

The API for Zone imports needs to be changed to be asynchronous. This would involve a new database table for zone import “statuses”, and a more robust system for managing the parsing and insertion of the zone data. Exporting zones will also return to the v2 API.

API Changes

GET /v2/zones/tasks/imports

This allows the user to view the statuses of their zone import requests. If the import has completed successfully, they can get the zone id and a link to the zone.

GET /v2/zones/tasks/import/cddda8f0-f558-11e3-a3ac-0800200c9a66 HTTP/1.1
Accept: application/json

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Location: /v2/zones/tasks/import/cddda8f0-f558-11e3-a3ac-0800200c9a66

{
  "imports": [
      {
        "id": "cddda8f0-f558-11e3-a3ac-0800200c9a66",
        "zone_id": "agqm44f0-s638-15e3-f4d3-1893572c9a67",
        "status": "SUCCESS",
        "links":{
            "self": "http://127.0.0.1:9001/v2/zones/tasks/import/cddda8f0-f558-11e3-a3ac-0800200c9a66",
            "href": "http://127.0.0.1:9001/v2/zones/agqm44f0-s638-15e3-f4d3-1893572c9a67"
      },
      {
        "id": "addda8f0-f558-11e3-a3ac-0800200c9a66",
        "zone_id": "qgqm44f0-s638-15e3-f4d3-1893572c9a67",
        "status": "SUCCESS",
        "links":{
            "self": "http://127.0.0.1:9001/v2/zones/tasks/import/addda8f0-f558-11e3-a3ac-0800200c9a66",
            "href": "http://127.0.0.1:9001/v2/zones/qgqm44f0-s638-15e3-f4d3-1893572c9a67"
      }
  ]
}

POST /v2/zones/tasks/imports

This creates a request to import a zone. The zone data is passed via the request body, and the Content-Type header must be set to ‘text/dns’.

This returns an id for the import request, for the user to query.

POST /v2/zones/tasks/import HTTP/1.1
Accept: application/json
Content-Type: text/dns

$ORIGIN example.com.
example.com. 42 IN SOA ns.example.com. nsadmin.example.com. 42 42 42 42 42
example.com. 42 IN NS ns.example.com.
example.com. 42 IN MX 10 mail.example.com.
ns.example.com. 42 IN A 10.0.0.1
mail.example.com. 42 IN A 10.0.0.2

HTTP/1.1 201 Accepted
Content-Type: application/json; charset=UTF-8
Location: /v2/zones/tasks/import/cddda8f0-f558-11e3-a3ac-0800200c9a66

{
    "id": "cddda8f0-f558-11e3-a3ac-0800200c9a66",
    "zone_id": null,
    "status": "PENDING",
    "links":{
        "self": "http://127.0.0.1:9001/v2/zones/tasks/import/cddda8f0-f558-11e3-a3ac-0800200c9a66"
    }
}

GET /v2/zones/tasks/imports/<id>

This allows the user to view the status of their zone import request. If the import has completed successfully, they can get the zone id and a link to the zone.

GET /v2/zones/tasks/import/cddda8f0-f558-11e3-a3ac-0800200c9a66 HTTP/1.1
Accept: application/json

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Location: /v2/zones/tasks/import/cddda8f0-f558-11e3-a3ac-0800200c9a66

{
    "id": "cddda8f0-f558-11e3-a3ac-0800200c9a66",
    "zone_id": "agqm44f0-s638-15e3-f4d3-1893572c9a67",
    "status": "SUCCESS",
    "links":{
        "self": "http://127.0.0.1:9001/v2/zones/tasks/import/cddda8f0-f558-11e3-a3ac-0800200c9a66",
        "href": "http://127.0.0.1:9001/v2/zones/agqm44f0-s638-15e3-f4d3-1893572c9a67"
    }
}

GET /v2/zones/<id>

This request, with the header “Accept:text/dns” exports the zone in DNS zonefile format to the user.

GET /v2/zones/cddda8f0-f558-11e3-a3ac-0800200c9a66 HTTP/1.1
Accept: text/dns

HTTP/1.1 200 OK
Content-Type: text/dns; charset=UTF-8
Location: /v2/zones/cddda8f0-f558-11e3-a3ac-0800200c9a66

HTTP/1.1 200 OK
Content-Type: text/dns

$ORIGIN example.com.
$TTL 42
example.com. IN SOA ns.designate.com. nsadmin.example.com. (
    1394213803 ; serial
    3600 ; refresh
    600 ; retry
    86400 ; expire
    3600 ; minimum
)
example.com. IN NS ns.designate.com.
example.com.  IN MX 10 mail.example.com.
ns.example.com.  IN A  10.0.0.1
mail.example.com.  IN A  10.0.0.2

Central Changes

create_import_domain(body)

Parameter	Description	Required
body	Unserialized request data from the API request	Yes

1. Create an entry in the zone_imports table to track the request with status PENDING. 2. Kicks off a thread to _import_domain with the request body. 3. Returns a zone_import object.

get_import_domain(id)

Parameter	Description	Required
import_id	The id of the zone import	Yes

1. Calls storage.find_import to get a specific zone_import record from the zone_imports table. 2. Returns the zone_import object

find_import_domains(context)

Parameter	Description	Required
context	Context to be passed to storage.find_imports	Yes

1. Calls off to the storage.find_imports to find all the zone imports for a tenant. 2. Returns the zone_import_list object

delete_import_domain(id)

Parameter	Description	Required
import_id	The id of the zone import	Yes

1. Calls off to storage.delete_zone_import to delete the zone import record from the zone_imports table.

_import_domain(body)

Parameter	Description	Required
body	Unserialized request data from the API request	Yes
zone_import	zone_import object from the original request	Yes

1. Tries to use dnspython’s from_text method to convert the zone into a dnspython object. If that doesn’t work, updates the zone_import object in storage to ERROR to indicate that the zone could not be imported. 2. Converts the dnspython zone to a Designate domain object. 3. Calls central.create_domain with the converted object. 4. Takes the return object’s id and updates the zone_import object in storage with the new zone_id and updates the status of the import to COMPLETE, or similar.

Storage Changes

A new table for zone import records will be added, along with the boilerplate get, set, delete, update methods.

New Table - zone_imports

Row	Type	Nullable?	Unique?	Notes
id	uuid	No	Yes	Primary key
zone_id	uuid	Yes	Yes	Zone id when the import completes
status	ENUM	Yes	No	One of [COMPLETE, PENDING, ERROR]
message	VARCHAR	Yes	No	A message letting the user know why their import failed. For example: “Malformed zonefile”, “Complete”

Other Changes

New DesignateObjects for the Imports will have be created, ZoneImport, ZoneImportList.

Exporting zones will also return to the v2 API, but should remain relatively unchanged.

Alternatives

Instead of creating a new zone_import table and object, it may be possible to create an empty domain object and call central._create_domain_in_storage and make an empty domain that is then updated with the result of the import. Only then could the call to the pool manager be made.

This might make the code simpler. But provides very little in the way of letting the user know that their import failed. You could add a status like MALFORMED_ZONEFILE or something, but that would still require the user to delete the zone before they tried again. Unless you soft-delete the zone when it fails and modify the default find_domains criterion to find zones that have been deleted only if they have that status.

Assignee(s)

Primary assignee:: tim-simmons-t

Milestones

Target Milestone for completion:: Liberty-1

Work Items

Implement the database table, migration.
Implement the storage methods.
Implement the central methods.
Fix the API to use the new code.
Move the APIs back into the v2 namespace

TSIG Signing for Worker Poll and Notify Messages

Tue, 17 Mar 2026 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://bugs.launchpad.net/designate/+bug/2142614

Designate’s worker service sends unsigned DNS NOTIFY and SOA poll queries to backend nameservers. In split-horizon DNS deployments using BIND views, this causes zone management failures because BIND uses TSIG keys to route requests to the correct view.

Problem description

When Designate manages zones across multiple BIND views (split-horizon DNS), each view is typically configured to accept queries and notifications based on match criteria such as source IP (match-clients), destination IP (match-destinations), or TSIG key. One option is to use TSIG keys, where BIND determines which view a DNS message belongs to by verifying the signature.

Currently, Designate’s MDNS service supports TSIG for AXFR (zone transfer) responses — BIND can successfully pull zone data from MDNS using TSIG-signed transfers. However, the worker service sends two types of DNS messages that are completely unsigned:

NOTIFY messages: After creating or updating a zone on a backend target, the worker sends a DNS NOTIFY to tell the nameserver to initiate an AXFR. Without TSIG, BIND either rejects the NOTIFY or routes it to the default (“any”) view if configured, which may be the wrong view.
SOA poll queries: After a zone change, the worker polls nameservers by sending SOA queries to verify the zone serial has been updated. Without TSIG, the SOA query hits the wrong view, returns the wrong serial (or NXDOMAIN), and the zone transitions to ERROR status.

This means that while Designate can technically create zones in BIND backends with view-based split-horizon, the zones inevitably transition to ERROR because the worker cannot properly notify or verify them.

The infrastructure to support TSIG signing partially exists:

The TsigKey object model stores TSIG key name, algorithm, and secret
The PoolTarget object has a tsigkey_id field
MDNS already uses TSIG for signing AXFR responses
dnspython provides Message.use_tsig() for signing outgoing queries

However, PoolNameserver (used for polling) has no TSIG reference, and the worker’s dnsutils.notify() / dnsutils.soa_query() functions have no TSIG support.

Proposed change

Add TSIG signing support to the worker’s NOTIFY and SOA poll messages, keyed by the pool target and nameserver configuration.

dnsutils Changes

Extend prepare_dns_message(), notify(), soa_query(), and send_dns_message() to optionally accept TSIG parameters and sign outgoing DNS messages.

When a TSIG key is provided, the DNS message is signed using dnspython’s Message.use_tsig() before being sent. This applies to both UDP and TCP transport.

The TSIG parameters (key name, secret, algorithm) are resolved from the TsigKey object referenced by the pool target or nameserver’s tsigkey_id.

Worker Changes

SendNotify: The SendNotify task already receives the pool target object. When target.tsigkey_id is set, the task looks up the TsigKey from storage and passes the TSIG parameters to dnsutils.notify().

PollForZone: The PollForZone task receives a PoolNameserver. With the addition of tsigkey_id to PoolNameserver (see below), the task looks up the TsigKey from storage and passes the TSIG parameters to dnsutils.soa_query().

ZonePoller: Passes the context to PollForZone so it can look up TSIG keys from storage.

API Changes

None. TSIG keys are configured via pool YAML and the existing TSIG key API.

Storage Changes

Add tsigkey_id column to the pool_nameservers table.

Altered Table - pool_nameservers

Row	Type	Nullable?	Unique?
tsigkey_id	uuid	Yes	No

This mirrors the existing tsigkey_id column on pool_targets.

A database migration will add this nullable column with a default of NULL, which preserves backwards compatibility — existing deployments without TSIG continue to work with unsigned queries.

Pool Configuration Changes

The pool YAML schema is extended to allow tsigkey_id on nameservers, consistent with the existing support on targets:

- name: internal-view-pool
  description: Pool for BIND internal view

  nameservers:
    - host: 192.0.2.1
      port: 53
      tsigkey_id: 4a1a4e28-2b4e-4b2a-8b0a-3c4d5e6f7a8b

  targets:
    - type: bind9
      description: BIND9 internal view
      tsigkey_id: 4a1a4e28-2b4e-4b2a-8b0a-3c4d5e6f7a8b

      masters:
        - host: 192.0.2.10
          port: 5354

      options:
        host: 192.0.2.1
        port: 53
        rndc_host: 192.0.2.1
        rndc_port: 953
        rndc_key_file: /etc/bind/rndc.key

- name: external-view-pool
  description: Pool for BIND external view

  nameservers:
    - host: 192.0.2.1
      port: 53
      tsigkey_id: 9b8c7d6e-5f4a-3b2c-1d0e-a9b8c7d6e5f4

  targets:
    - type: bind9
      description: BIND9 external view
      tsigkey_id: 9b8c7d6e-5f4a-3b2c-1d0e-a9b8c7d6e5f4

      masters:
        - host: 192.0.2.10
          port: 5354

      options:
        host: 192.0.2.1
        port: 53
        rndc_host: 192.0.2.1
        rndc_port: 953
        rndc_key_file: /etc/bind/rndc.key

In this configuration, both pools point to the same physical BIND server (192.0.2.1), but use different TSIG keys to route to different views. The nameserver and target for the same view typically share the same TSIG key.

The TSIG keys must be created beforehand via the Designate TSIG key API with scope POOL and the corresponding pool’s resource_id.

Other Changes

PoolNameserver object: Add tsigkey_id field (UUIDFields, nullable).

Pool update command: designate-manage pool update already reads pool YAML and updates pool objects. It must handle the new tsigkey_id field on nameservers, same as it already does for targets.

Alternatives

TSIG key on pool (not target/nameserver): Use a single TSIG key per pool instead of per-target and per-nameserver. Simpler, but doesn’t work when a pool has multiple targets or nameservers that each require their own key (e.g., multiple BIND servers with distinct TSIG keys). Keeping TSIG per-target and per-nameserver is consistent with the existing target-level tsigkey_id and gives operators full control.
Derive nameserver TSIG from target TSIG: Instead of adding tsigkey_id to nameservers, infer the TSIG key from the target that shares the same host. This avoids a schema change but creates a fragile implicit coupling — the pool data model has no explicit relationship between targets and nameservers, so matching would require comparing hosts across different schemas (nameserver host vs. target options.host), with the risk of ambiguous or missing matches.
Configuration file options: Add TSIG key name/secret/algorithm as worker configuration options (e.g., [service:worker] tsig_key_name). This would apply a single TSIG key to all poll/notify messages globally, which doesn’t work for multi-pool split-horizon deployments where each pool needs a different key.

Implementation

Assignee(s)

Primary assignee:: Omer Schwartz (oschwart)

Milestones

Target Milestone for completion:: 2026.2 (Hibiscus)

Work Items

Add tsigkey_id to PoolNameserver object model
Database migration: add tsigkey_id column to pool_nameservers table
Extend dnsutils.notify() and dnsutils.soa_query() to accept and apply TSIG parameters
Update SendNotify to look up and use TSIG key from target
Update PollForZone / ZonePoller to look up and use TSIG key from nameserver
Update designate-manage pool update to handle nameserver tsigkey_id
Unit and functional tests
Documentation: pool configuration guide for split-horizon with TSIG

Dependencies

None. All required libraries (dnspython TSIG support) and infrastructure (TsigKey model, PoolTarget.tsigkey_id) already exist.

Upgrade Implications

A database migration adds the nullable tsigkey_id column to pool_nameservers. Existing deployments are unaffected — the column defaults to NULL, and when no TSIG key is configured, the worker continues to send unsigned messages as it does today. No changes to existing pool YAML files are required.

Zone Attributes Support for Zone Import

Tue, 17 Mar 2026 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://bugs.launchpad.net/designate/+bug/1690184

Zone import currently does not support specifying zone attributes. This means zones imported via POST /v2/zones/tasks/imports are always scheduled to the default pool. With the deprecation of pool management APIs, zone attributes are the primary mechanism for controlling pool assignment during zone creation. Zone import should support the same attributes that zone creation does, so that imported zones can be scheduled to the correct pool.

Problem description

When creating a zone via POST /v2/zones, users can specify attributes in the JSON body to influence pool scheduling. For example:

{
    "name": "example.com.",
    "email": "admin@example.com",
    "attributes": {"pool_id": "794ccc2c-d751-44fe-b57f-8894c9f5c842"}
}

The pool scheduler uses these attributes to match the zone to the appropriate pool. However, zone import (POST /v2/zones/tasks/imports) only accepts a text/dns body containing the raw zonefile. There is no way to pass zone attributes, so imported zones always land in the default pool.

This is a significant limitation for operators managing multiple pools. The available workarounds are cumbersome: one option is to import each zone into the default pool and then use zone pool move to relocate each zone individually to the correct pool — a tedious per-zone process. Another option is to skip import entirely, create each zone via POST /v2/zones with the desired attributes, and then populate it with records through the API — which defeats the purpose of having a zonefile-based import. With pool management APIs deprecated, this gap becomes even more critical, as attributes are now the standard way to control pool assignment.

Proposed change

Extend the zone import endpoint to optionally accept application/json as a content type, in addition to the existing text/dns. When the request body is JSON, it may include both the zonefile content and zone attributes:

{
    "zonefile": "$ORIGIN example.com.\n$TTL 3600\n...",
    "attributes": {"pool_id": "794ccc2c-d751-44fe-b57f-8894c9f5c842"}
}

The attributes are passed through to the zone object before creation, using the same code path as POST /v2/zones. This means pool scheduling, validation, and attribute storage all work identically to normal zone creation.

When no attributes are provided, or when the request uses text/dns, the behavior is unchanged — the zone is scheduled to the default pool as it is today. This makes the change fully backwards compatible.

API Changes

POST /v2/zones/tasks/imports

The endpoint currently accepts only text/dns. This change adds application/json as an accepted content type.

Existing behavior (unchanged):

Request:

POST /v2/zones/tasks/imports HTTP/1.1
Content-Type: text/dns

$ORIGIN example.com.
$TTL 3600
example.com. IN SOA ns1.example.com. admin.example.com. (
    2024101401 3600 900 1209600 300 )
example.com. IN NS ns1.example.com.
example.com. IN A 192.0.2.1

Response:

HTTP/1.1 202 Accepted
Content-Type: application/json

{
    "id": "074e2e30-8748-11e4-a4a2-08002783c8f3",
    "status": "PENDING",
    "message": null,
    "zone_id": null,
    "links": {
        "self": "/v2/zones/tasks/imports/074e2e30-8748-11e4-a4a2-08002783c8f3"
    }
}

New behavior (with attributes):

Request:

POST /v2/zones/tasks/imports HTTP/1.1
Content-Type: application/json

{
    "zonefile": "$ORIGIN example.com.\n$TTL 3600\nexample.com. IN SOA ...",
    "attributes": {
        "pool_id": "794ccc2c-d751-44fe-b57f-8894c9f5c842"
    }
}

Response:

HTTP/1.1 202 Accepted
Content-Type: application/json

{
    "id": "074e2e30-8748-11e4-a4a2-08002783c8f3",
    "status": "PENDING",
    "message": null,
    "zone_id": null,
    "links": {
        "self": "/v2/zones/tasks/imports/074e2e30-8748-11e4-a4a2-08002783c8f3"
    }
}

JSON body parameters:

Parameter	Description	Required
zonefile	The zonefile content as a string (same format as the text/dns body).	Yes
attributes	Zone attributes as key-value pairs. Accepts the same attributes as zone creation (e.g. pool_id, scope).	No

Error cases:

application/json body missing zonefile field: 400 Bad Request
Invalid attributes (e.g. referencing non-existent pool): handled during zone creation, reported in the zone import status as ERROR

Central Changes

The create_zone_import method in central will accept an optional zone_attributes dict parameter. This is passed through to _import_zone, which sets the attributes on the zone object before calling create_zone.

The attributes are parsed using the existing ZoneAttributeListAPIv2Adapter, the same adapter used by zone creation. This ensures consistent validation and behavior.

Storage Changes

None. Zone attributes are already stored via the existing zone creation flow.

Other Changes

python-designateclient

The openstack zone import create command will accept an optional --attributes flag using the same key:value format as openstack zone create:

openstack zone import create /path/to/zonefile \
    --attributes pool_id:794ccc2c-d751-44fe-b57f-8894c9f5c842

When --attributes is provided, the client sends the request as application/json with the zonefile content and attributes in the body. When --attributes is omitted, the client sends text/dns as it does today.

openstacksdk

Update the ZoneImport resource in openstacksdk to support the new application/json content type and zone attributes.

designate-tempest-plugin

API tests covering:

Zone import with attributes (verify pool assignment)
Zone import with application/json content type
Zone import without attributes (backwards compatibility)
Zone import with invalid attributes (error handling)

api-ref

Update the zone import API reference documentation to describe the new application/json content type and the attributes parameter.

Alternatives

Several alternatives were considered:

Custom HTTP header (X-Designate-Zone-Attributes): Pass attributes as a custom header. This was prototyped in https://review.opendev.org/c/openstack/designate/+/932323 but rejected because headers are not easily discoverable in API documentation, require fragile string-to-JSON parsing, and need CORS configuration changes.
Query parameters: Pass attributes as query parameters (e.g. ?attributes=pool_id:uuid). Simple for a single attribute but becomes unwieldy with many attributes and is inconsistent with how zone creation accepts them.
Two-step process: Import the zone, then update attributes via PATCH. Not viable because pool scheduling happens at creation time — the zone would already be assigned to the wrong pool.

The JSON body approach was chosen because it reuses the exact same attribute format as zone creation, scales to any number of attributes, and is easy to document and discover.

Implementation

Assignee(s)

Primary assignee:: Omer Schwartz (oschwart)

Milestones

Target Milestone for completion:: 2026.2 (Hibiscus)

Work Items

Designate: extend zone import endpoint to accept application/json
Designate: pass attributes through central to zone creation
Designate: unit and functional tests
python-designateclient: add --attributes flag to zone import create
openstacksdk: update ZoneImport resource to support attributes
designate-tempest-plugin: API tests for zone import with attributes
api-ref: document new content type and attributes parameter

Dependencies

None. This feature builds entirely on existing zone attribute and pool scheduling infrastructure.

Upgrade Implications

None. The change is backwards compatible. Existing text/dns requests continue to work unchanged. The new application/json content type is opt-in.

Catalog Zones

Fri, 08 Jul 2022 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

DNS Catalog Zones is a method in which a catalog of zones is represented as a regular DNS zone, and transferred using DNS zone transfers. As zones are added to or removed from the catalog zone, the changes are propagated to the secondary nameservers in the normal way. The secondary nameservers then add/remove/modify the zones they serve in accordance with the changes to the zone.

(https://zones.cat/)

Problem description

Currently Designate does not provide a list of zones for secondary nameservers to consume and to be provisioned from. Rather Designate manually creates, modifies and deleted every zone one by one via a call to a management tooling like BIND’s rndc. This mechanism has scaling issues for large number of domains being added and maintained on secondary server and a slow convergence in case of new servers being added to the server pool.

Proposed change

Support for providing catalog zones should be added to Designate.

In essence catalog zones provide easy provisioning capabilities of lists of zones on secondary servers via AXFR from a special zone, the catalog.

The below links explain the catalog zone feature, use cases and data model:

There already is support in BIND9, Knot and other major DNS servers like PowerDNS or NSD to consume catalog zones, all of them are backends Designate supports:

Apart from improving on performance and resilience of the provisioning in general, supporting catalog zones also enables Designate to support other backend implementationsas secondary DNS servers without the requirement for dedicated drivers.

API Changes

None

Central Changes

The Pool object (and maybe adapters) will need to updated to include the new pool settings required for creating a catalog zone. See the Other Changes section for details of these settings.
Central must be updated to increment the pool catalog zone serial number when a zone is created, updated, or deleted in the associated pool. It must also trigger a NOTIFY to the nameservers in the affected pool.
Central and/or the workers must be updated to not directly create the zones on the backend DNS servers if the pool has a catalog zone configured.

Storage Changes

The zone database schema will need to be updated to add a new “type” of CATALOG to indicate that this zone is a catalog zone.
The existing zone “version” field can be used to track the catalog zone “Schema Version” which is required to be present in a TXT record for the zone. As of this spec, the version must be “2”.
The catalog zone will not have an associated tenant_id.
A new storage query will need to be created that merges the zone SOA and NS records with a the list of zone PTR records for the catalog.
1. The zone records returned for the catalog will use the zone ID as the “unique ID” for the zone PTR record in the catalog.
Storage create_pool and update_pool will need to be updated to create the catalog zone for the pool when the proper settings are included in the pools.yaml. See the Other Changes section for details of these settings.
Storage create_pool and update_pool will need to be updated to create or update the catalog zone TSIG key record in the database when one is provided in the pools.yaml “catalog_zone_tsig_key” setting.
Storage create_pool and update_pool will need to be updated to capture the catalog zone attributes in the existing pool_attributes table. See the Other Changes section for details of these settings.

Other Changes

The pool definition will next to be extended to add an optional section “catalog_zone”.
1. A “catalog_zone_fqdn” key/value will be added below the “catalog_zone” parent. This will be a required field if “catalog_zone” is specified. It will be validated to ensure it is a valid Fully Qualified Domain Name (fqdn).
2. A “catalog_zone_refresh” key/value will be added below the “catalog_zone” parent. This is an optional setting that will use a default of 60 (seconds) if the parameter is not specified. This value will be validated to be in the range of 0 to 2147483647 as with other zones.
3. A “catalog_zone_tsig_key” key/value will be added below the “catalog_zone” parent. This setting is optional, but will be highly recommended in the documentation for this feature. It will be validated the same way TSIG keys are validated in the API.
4. A “catalog_zone_tsig_algorithm” key/value will be added below the “catalog_zone” parent. This setting must be specified if a “catalog_zone_tsig_key” is specified. It will be validated the same way the TSIG algorithm is validated in the API.
Documentation must be created for setting up catalog zones.
1. It must include a security warning and guide to adding a TSIG key to the catalog zone.
When the catalog zone is created by a pools.yaml pool creation, only one NS record will be created for the zone. It must contain “invalid.” as the NSDNAME as recommended in the catalog zone specification.
The catalog zone SOA record fields: RETRY and EXPIRE will be defined the same way other Designate zones are defined.
Tests must be updated to cover this new capability.
1. This should include tests that validate that non-admin users cannot see the catalog zones.
A release note must be created for the feature.

Other Notes

This spec does not include creating “masters” records in the catalog zone. This extension does not seem to be finalized at this time and it does not currently support alternate port numbers. This could be added at a future time.
This spec does not include allowing backend drivers to configure the catalog zones on the backend DNS servers. This could be added at a future time.
The catalog zone TSIG key is included in the pools.yaml as opposed to created via the API to ensure the zone is protected as soon as possible and to facilitate future patches that may create the catalog zone consumer on the backend DNS servers automatically.

Alternatives

The primary alternative to implementing “catalog zones” is the current mechanism used by the Designate worker to manage zones on the backend name servers. Examples of this are: for BIND, rndc zone management; for PowerDNS, the HTTP API.

Providing the means to allow backends to pull a list of their zones first of all provides an alternative or addition to the explicit configuration of backends by the worker and their drivers.

While there is the Designate API available to fetch zone data for other means of provisioning backends, the catalog zone provides a common and agreed interface to make this data available via AXFR and to various DNS servers.

There is no real alterative to providing a catalog zone, as it is a distinct data model and protocol to be then provided by Designate.

Implementation

Assignee(s)

Primary assignee:: Christian Rohmann https://launchpad.net/~christian-rohmann
IRC Nick Name:: crohmann

Milestones

Work Items

Dependencies

Upgrade Implications

Designate Zone Pool Move

Fri, 04 Jun 2021 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/zone-move

Zone move is an important feature that possibly eliminates the problem observed with zone export-import.

Problem description

Assume, zone is currently located in pool A, but has to move to pool B. (for example pool B is another, new DNS provider).

Currently, Designate only allows you to create zone export (will be a Bind-format single file with all the records of the zone) and zone import. So, a zone can be exported from one pool; deleted from pool A; and imported into another pool B.

The problem is that the zone has to be deleted in this process and imported.
Large zone imports (20-30k records) can take a couple of hours as Designate creates all records one by one.

Therefore, it is problematic to move a zone between the pools/backends without any hacks. At the point when the zone is deleted, but not imported, any DNS queries to Designate-MDNS from the backend will not work.

Proposed change

Introduce a new admin-only command in designate zone pool move. The command works as below:

openstack zone pool move zone_id_or_name –pool_id=AN_ID_OF_POOL_B

The designate service should update zone pool_id property similar to other zone properties. The updated pool_id will be reflected in the designate DB. If the command is invoked without pool_id paramter, the designate reschedule the zone and move it to a different pool if appropriate. For example, user can move their zone up to a higher tier of service by updating an attribute and then calling move will cause attribute scheduler to select appropriate pool.
After DB update, designate will create copy of zone (can also be called as clone) except it should not create new db entry for the clone zone. The clone zone will be created on target pool backend servers i.e. pool B.
The zone transfer(AXFR/IXFR) will happen and the zone on pool B gets synced with the designate DB.
At this point zone still exists in pool A. It can be removed after the administrator has changed the settings manually at domain registrar. This is a manual process.

The above proposed change eliminates the zone export as well as zone delete steps. Thus, speed up the import of large zone to the target pool.

API Changes

The zone move will be triggered via the V2 API.

Example zone pool move request:

POST http://192.168.1.47/dns/v2/zones/52f42b9e-c48b-43e2-af01-180e8ed33cd0/tasks/pool_move HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json
Content-Type: application/json

{
  "pool_id": "794ccc2c-d751-44fe-b57f-8894c9f5c842"
}

Central Changes

Central will update the pool_id of the zone for which move is requested. The updated pool_id will be written to DB. Central will then clone the zone and then create new clone zone in target pool backend servers.

Implementation

Assignee(s)

Primary assignee:: Kiran Pawar (kpdev in Gerrit)

Dependencies

None

A Worker Model for Designate

Wed, 03 Mar 2021 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

Thesis: Gratuitous unnecessary complexity exists within the Designate code, and in the operation of Designate as a service. Making Designate a producer-worker type of project will vastly simplify the development and operation, and align it more to the true nature of the service it provides (DNS).

Problem description

designate-pool-manager does a reasonably good job at pushing Create/Delete changes out to nameservers, but the process gets a bit less shiny after that.

Polling to see if state is live is done via asynchronous and synchronous RPC with another component.
Cache usage is a mess (storing mutliple keys, keeping one key around forever, storing a different key for each type of operation).
Periodic Sync/Recovery are very unreliable as the number of changes grows.
The update_status and logic for calculating consensus is heavy-handed, too eager, and too complex.
The state machine is very foggy, and the logic for updating status that gets pushed into central is obfuscated.
Pool Managers are tied to one pool

designate-zone-manager does a good job at executing periodic timers for the zones it manages. However:

One zone-manager process is responsible for a certain set of zones, if the operations for that set of zones gets heavy, a single zone manager process could become overwhelmed.
We rely on tooz to manage the extremely delicate task of ensuring balance and coverage of all zones by zone manager processes.
Certain work (export) that’s in the critical path of operations has already crept into the component that wasn’t really meant for that. As a substitute for proper workers, the zone-manager is looking like the current answer.

designate-mdns is a DNS server written in Python. It works well for small amounts of traffic, but as traffic grows, we may realize that we need it to be more specialized, as a DNS server written in Python should be. The logic for sending NOTIFYs and polling of changes seems less likely to belong in mdns in the future. If those bits were removed, designate-mdns could be rewritten to make use of a better tool for the problem.

Proposed change

A change to the underlying architecture of executing actual work on DNS servers and the running of other tasks. Essentially, removing designate-pool-manager and designate-zone-manager, replacing them with designate-worker and designate-producer (names up for debate) and removing certain logic from designate-mdns. All of the actual “work” would be put in the scalable designate-worker process, which has work produced by the API/Central, and designate-producer. designate-mdns gets back to it’s roots, and only answers AXFRs. Callbacks over queues that don’t involve the API are eliminated, simplifying the code in all components that deal with DNS servers.

No changes to the API or Database are required, with minimal changes to designate-central.

To the end user, the results of this change would be relatively simple:

Scalability limited only by DNS servers, datastores, and queues. If at any point Designate starts to slow down in some aspect, unless there’s an issue with those services listed above, the problem can be solved by throwing more Designate processes at the problem.
Fault tolerance. One or more Designate processes dying would be inivisible in almost every case. An operator wouldn’t have to fear a certain process dying, because no one process handles responsiblities others will not, as long as there is a minimal amount of redundancy.
Simplicity. The Designate architecture becomes much cleaner and easier to understand. The questions of “what is the difference between pool-manager and zone-manager” and “what’s synchronous and asynchronous” become less muddy, or disappear altogether.
Operationally, this opens the door for simpler scenarios for small deployments where a customer need only scale a couple components (even on the same machine) to get more performance. You could even go so far as to deploy app nodes that only shared datastore (db, cache) and had their own designate components and queues.

Architectural Changes

These are the services that would remain present:

designate-api - To receive JSON and parse it for Designate
designate-central - To do validation/storage of zone/record data and send CRUD tasks to designate-worker.
designate-mdns - To only perfrom AXFRs from Designate’s database
designate-worker - Any and all tasks that Designate needs to produce state on nameservers
designate-producer - To run periodic/timed jobs, and produce work for designate-worker that is out the normal path of API operations. For example: Periodic recovery.

Other necessary components:

Queue - Usually RabbitMQ
Database - Usually MySQL
Cache - (encouraged, although not necessary) Memcached, MySQL, Redis
A tooz backend (Zookeeper, Memcahed, Redis)

Services/components that are no longer required:

designate-pool-manager
designate-zone-manager

Designate Worker

The scope of designate-worker’s duties are essentially any and all tasks that Designate needs to take action to perform. For example:

Create, Update, and Delete zones on pool targets via backend plugins
Poll that the change is live
Update a cache with the serial number for a zone/target
Emit zone-exists events for billing
Flatten Alias Records
Clean up deleted zones
Importing/Exporting zones
Many more

The service essentially exposes a vast RPCAPI that contains tasks.

An important difference to Designate’s current model is that all of these tasks do not call back. They are all fire-and-forget tasks that will be shoved on a queue and await worker action.

tasks are essentially functions, that given relatively simple input, make the desired income happen on either nameservers, or the Designate database.

Cache

The cache performs a similar function to the current pool manager cache now.

It will store state for each different type of task that a worker can use to decide if it needs to continue with a task received from the queue, or simply drop it and move on to the next task.

This varies by task, some are relatively simple, knowing whether to perform a zone update to a certain serial number is knowable by seeing the serial number of a zone on each target in a pool. For DNSSEC zone signing, a key would probably be placed to indicate that a certain worker was working on resigning a zone, as it’s a more long-running process.

In the absence of such a cache, each worker will act naive and try to complete each task it receives.

Tasks

Each task will be idempotent, to the degree that it is possible.

As mentioned in the Cache section, to a certain degree, tasks could be able to know if they need to complete work based on information in the cache.

But they should also make an effort to not duplicate work, for instance, if it’s trying to delete a zone that’s already gone, it should interpret the zone being gone as a sign that the delete is successful and move on.

On the whole these tasks would simply be lifted from where they currently exist in the code, and wouldn’t change all that much.

A slight change might be that during the course of the task, we may recheck that the work that is being undertaken still needs to be done.

As an example: An API customer creates many recordsets very quickly. The work being dispatched to designate-worker processes would go to a lot of different places, and one of the first updates to actually reach a nameserver might contain all the changes necessary to bring the zone up-to-date. The other tasks being worked should check before they send their NOTIFY that the state is still behind, and check again after they’ve sent their NOTIFY, but before they’ve began polling, so that they can cut down on unnecessary work for themselves, and the nameservers.

You could get even smarter about the markers that you drop in a cache for these tasks. For example, on a zone update, you could drop a key in the cache of the nature zoneupdate-foo.com., and other if other zoneupdate tasks for the same zone see that key, they could know to throw away their job and move on.

designate-mdns changes

The partioning of certain elements that Designate had previously disappears. The worker service will send DNS queries, it will do cpu-bound tasks, but it will be one place to scale. It should be possible to have an extremely robust Designate architecture by simply scaling these workers.

designate-mdns will have it’s entire RPCAPI transferred to designate-worker. This will vastly simplify the amount of work it needs to do while it sits in the critical path of providing zone transfers to nameservers Designate manages.

As a side-note, this would make this service much easier to optimize, or even rewrite in a faster programming language.

Designate Producer

designate-producer is the place where jobs that produce tasks that are outside of the normal path of API operations and operate on some kind of timer live.

The key difference to the zone-manager service, is that this service simply generates work to be done, rather than actually doing the work. designate-producer simply decides what needs to be done, and sends RPC messages on the queue to designate-worker to actually perform the work.

As we’ve grown Designate, we’ve seen the need for this grow vastly, and even more so in the future.

Deleted zone purging
Refreshing Secondary Zones
Emitting zone exists tasks and other billing events
DNSSEC signing of zones
Alias record flattening

We could move the periodic_sync and periodic_recovery tasks from the Pool Manager to this service.

The periodic_sync and periodic_recovery tasks in the Pool Manager have been a constant struggle to maintain and get right. This is due to a lot of factors.

Making the generation of tasks by periodic processes the job of only one Designate component simplifies the architecture, and allows to solve the problems it presents one time, one way, and generally do one thing well.

Timers

This service would essentially be a group of timers that wake up on a cadence and create work to be put on the queue for designate-worker processes to pick up.

The overhead is relatively low here, as we’re not actually doing the work, but more just scheduling the work to be done. This way we can focus on the unexpectedly difficult problem of dividing up the production of work that these processes will put on the queue.

Dividing work

To explain more clearly, the biggest problem we have in this service is making it fault-tolerant, but not duplicating work for designate-worker processes to do. This was solved before by tooz using the zone shards in the Designate database in designate-zone-manager and it seems to work well.

designate-worker processes, as described above, will do a certain amount of optimization so that they don’t duplicate work. But if we generate too much cruft, those processes will be bogged down just by the task of seeing if they need to do work. So we should work to minimize the amount of duplicate work we produce.

Queue Priority

One potential complication of this implementation is that, as the number of timers and tasks that are out of Designate’s critical path of implementation grow, they may get in the way of designate-worker processes doing the tasks that are most important, namely CRUD of zones and records.

We propose having queues/exchanges for each type of task, this would be an optimal way to monitor the health of different types of tasks, and isolate the sometimes long-running tasks that periodic timers will produce from the relatively quicker, and more important CRUD operations. The algorithm for choosing tasks from the various options could be customized by a particular process if desired. But a good general default would be to handle CRUD operations from designate-central first. Or use a weighted random choice algorithm, with the critical-path CRUD operations having higher weights.

Work Items

Stand up a designate-worker service
Migrate CRUD zone operations to designate-worker, reworking the cache implementation.
Stand up designate-producer service
Migrate Pool Manager periodic tasks to designate-producer, with small modifications to ensure they simply generate work for designate-worker
Move designate-mdns’ NOTIFYing and polling to designate-worker
Fix up the update_status logic in designate-central
Migrate all tasks from zone-manager to a split of designate-worker and designate-producer, where producer creates the work on the queue and worker executes it. Ensuring scalable logic for distributed work production using cache or some other method in designate-producer
Deprecate pool-manager and zone-manager
Profit!!!

Upgrade Implications

Upgrading to the next release with this change would introduce some operational changes. Mostly around the services that need to be deployed. The deployment need not be a cutover, deploying Newton Designate will work with or without the worker. This is because of a variety of compatibility measures taken:

designate-central will have a configurable “zone api” that it can swap between designate-pool-manager and designate-worker. If the worker process is enabled, central can send c/u/d zones events to the worker instead of the pool manager.
designate-worker’s ability to send NOTIFYs and Poll DNS servers can replace a portion of designate-mdns’ responsibilities. For certain DNS servers, it’s theorized that they won’t behave well if a NOTIFY comes from a different server than it’s master that it zone transfers from. For this reason, designate-worker’s ability to send NOTIFYs is a configurable element. Since the worker calls into a backend plugin to update zones, the NOTIFY via mdns logic in those backends can remain, and if the operator so chooses, the NOTIFY task in the worker can no-op. This works both ways. An operator can also choose to have the MiniDNS notify calls noop, and allow them to be completed by the worker process.
For those who choose to firewall all DNS traffic between Designate and DNS servers, it will be safest to deploy designate-worker processes in close proximity to designate-mdns processes. So that the DNS polling that designate-worker does can be completed, where designate-mdns used to do it.
As periodic-type processes are migrated to designate-producer and designate-worker, they can be marked as “worker tasks” in designate-zone-manager, that can be turned off behind a configuration flag.

The process for upgrading to the worker model code, after deploying Newton could look something like this:

(To account for firewalling dns traffic) Start designate-worker processes operating on the same IPs as designate-mdns processes via proximity or proxy. The default configuration will still allow NOTIFYs and DNS polling to occur via designate-mdns, and all other operations to work in designate-pool-manager. No traffic will reach the worker.
Toggle configuration values designate-worker::enabled and designate-worker::notify and restart designate-worker.
Restart designate-central and designate-mdns processes so that mdns NOTIFY calls no-op, and central starts to use the worker instead of designate-pool-manager.
Toggle the designate-zone-manager::worker-tasks config flag and restart designate-zone-manager so that it hands off periodic tasks to the producer/worker.
Start the designate-producer process so that the worker starts doing recovery and other periodic tasks.
Stop the designate-pool-manger processes, and if all processes are migrated out of designate-zone-manager, that as well.

Milestones

Target Milestone for completion:: Newton

Author(s)

Tim Simmons https://launchpad.net/~timsim

Paul Glass https://launchpad.net/~pnglass

Eric Larson https://launchpad.net/~eric-larson

Filtering API

Wed, 03 Jun 2020 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

Discussion Needed

Update:

For now, wildcards will not be restricted, though limitations may have to be added due to scaling concerns. Also, to address a bug with wildcard matching (https://bugs.launchpad.net/designate/+bug/1335902) and to conform to standards, the wildcard character has been changed from ‘%’ to ‘*’.

Previous Discussion:

http://paste.openstack.org/show/85431/

Problem description

https://blueprints.launchpad.net/designate/+spec/example

Filtering provides the ability to qualify the result set returned by a query to the designate api. It will ultimately be available on all collections - zones, record sets, rdata of record sets, and pools.

Filtering will be controlled using query parameters which match the name of the attribute being filtered. It is not required that all attributes are available as filter targets, but the majority will be.

Filters are either an exact match or wildcard match, which is specified by the presence of the reserved wildcard character (‘*’).

If the filtering request is successful, the resources that pass the filter criteria are returned, as well as links for retrieving more details.

Pagination of results will take advantage of proposed Designate pagination (https://blueprints.launchpad.net/designate/+spec/pagination)

Proposed change

Filtering vs. Search: A Clarification

NOTE: Filtering and searching are two completely different features and will be addressed separately. Search involves the ability to compile a list of results from storage, possibly drawing from many different places (for example, finding all of a tenant’s A records with a certain IP address). Filtering only involves further restricting the standard queries that are offered by the API (for example, /zones, /zones/{id}/recordsets, etc.).

You can find the wiki for search here

API Changes

Basic filtering for:
- Blacklists: pattern
- Recordsets: name, type, ttl, data
- TLDs: name
- Zones: name
Wildcard search using SQL pattern matching

The examples below demonstrate the types of calls typically made with filtering. These calls apply to all of the aforementioned filtering parameters.

Verb	Resource	Description
GET	zones/{id}/recordsets/{id}/recordsets? name=www.example.com	From the specified recordset, return any record with a name that matches the specified filter (“name”) exactly.
GET	zones/{id}/recordsets/{id}/recordsets? name=www.test*	From the specified recordset, return any record with a name that BEGINS with the specified “name” attribute.
GET	zones/{id}/recordsets/{id}/recordsets? name=*.com	From the specified recordset, return any record with a name that ENDS with the specified “name” attribute.
GET	zones/{id}/recordsets/{id}/recordsets? name=www.*.com	From the specified recordset, return any record with a name that contains the specified name data on both sides of the wildcard.

GET zones/{zone-id}/recordsets?name=example*

This call is for a privileged user filtering recordsets by name, for one tenant, using wildcard matching.

Request:

GET zones/{zone-id}/recordsets?name=example*
Host: dns.provider.com
Accept: application/json

Response:

{
    "recordsets": [
        {
            "description": null,
            "links": {
                "self": "http://192.168.33.8:9001/v2/zones/a4e29ed3-d7a4-4e4d-945d-ce64678d3b94/recordsets/948a0233-abdf-434e-a003-c5d682daf0ea"
            },
            "updated_at": null,
            "records": [],
            "ttl": null,
            "id": "948a0233-abdf-434e-a003-c5d682daf0ea",
            "name": "example.com.",
            "zone_id": "a4e29ed3-d7a4-4e4d-945d-ce64678d3b94",
            "created_at": "2014-07-08T20:28:27.000000",
            "priority": [],
            "version": 1,
            "type": "NS"
        },
        {
            "description": null,
            "links": {
                "self": "http://192.168.33.8:9001/v2/zones/a4e29ed3-d7a4-4e4d-945d-ce64678d3b94/recordsets/7da6119a-8b41-4f09-a2b7-a44ed2c9ebd0"
            },
            "updated_at": null,
            "records": [
                "mail2.example.org.",
                "mail.example.org."
            ],
            "ttl": null,
            "id": "7da6119a-8b41-4f09-a2b7-a44ed2c9ebd0",
            "name": "example.org.",
            "zone_id": "a4e29ed3-d7a4-4e4d-945d-ce64678d3b94",
            "created_at": "2014-07-08T20:28:28.000000",
            "priority": [
                20,
                10
            ],
            "version": 1,
            "type": "MX"
        }
    ],
    "links": {
        "self": "http://192.168.33.8:9001/v2/zones/a4e29ed3-d7a4-4e4d-945d-ce64678d3b94/recordsets?name=example%2A"
    }
}

GET zones/{zone-id}/recordsets?name=example.org.

This call is for customers filtering recordsets by name, using exact matching.

Request:

GET zones/{zone-id}/recordsets?name=www.example.org.
Host: dns.provider.com
Accept: application/json

Response:

{
    "recordsets": [
        {
            "description": null,
            "links": {
                "self": "http://192.168.33.8:9001/v2/zones/a4e29ed3-d7a4-4e4d-945d-ce64678d3b94/recordsets/7da6119a-8b41-4f09-a2b7-a44ed2c9ebd0"
            },
            "updated_at": null,
            "records": [
                "mail2.example.org.",
                "mail.example.org."
            ],
            "ttl": null,
            "id": "7da6119a-8b41-4f09-a2b7-a44ed2c9ebd0",
            "name": "example.org.",
            "zone_id": "a4e29ed3-d7a4-4e4d-945d-ce64678d3b94",
            "created_at": "2014-07-08T20:28:28.000000",
            "priority": [
                20,
                10
            ],
            "version": 1,
            "type": "MX"
        }
    ],
    "links": {
        "self": "http://192.168.33.8:9001/v2/zones/a4e29ed3-d7a4-4e4d-945d-ce64678d3b94/recordsets?name=example%2A"
    }
}

GET zones/{zone-id}/recordsets?data=1.2.3.*

This call is for customers who wish to find all recordsets in a zone that contains one or more records with the matching data value (with a wildcard applied). The data parameter can be used in conjunction with other parameters.

Request:

GET zones/{zone-id}/recordsets?data=1.2.3.*
Host: dns.provider.com
Accept: application/json

Response:

{
    "recordsets": [
        {
            "description": null,
            "links": {
                "self": "http://192.168.33.8:9001/v2/zones/a4e29ed3-d7a4-4e4d-945d-ce64678d3b94/recordsets/077ce2b4-1d52-4c3f-8c70-07dfddeed5cc"
            },
            "updated_at": null,
            "records": [
                "1.2.3.4"
            ],
            "ttl": null,
            "id": "077ce2b4-1d52-4c3f-8c70-07dfddeed5cc",
            "name": "dns2.example.com.",
            "zone_id": "a4e29ed3-d7a4-4e4d-945d-ce64678d3b94",
            "created_at": "2014-07-08T20:28:26.000000",
            "priority": [
                null
            ],
            "version": 1,
            "type": "A"
        },
        {
            "description": null,
            "links": {
                "self": "http://192.168.33.8:9001/v2/zones/a4e29ed3-d7a4-4e4d-945d-ce64678d3b94/recordsets/758004fe-4f0f-4756-a8f3-ee01ca5db8a2"
            },
            "updated_at": null,
            "records": [
                "1.2.3.10"
            ],
            "ttl": null,
            "id": "758004fe-4f0f-4756-a8f3-ee01ca5db8a2",
            "name": "mail2.example.com.",
            "zone_id": "a4e29ed3-d7a4-4e4d-945d-ce64678d3b94",
            "created_at": "2014-07-08T20:28:20.000000",
            "priority": [
                null
            ],
            "version": 1,
            "type": "A"
        }
    ],
    "links": {
        "self": "http://192.168.33.8:9001/v2/zones/a4e29ed3-d7a4-4e4d-945d-ce64678d3b94/recordsets?data=1.2.3.*"
    }
}

Central Changes

Some restrictions on the usages of wildcard filtering may be applied in the future.

Storage Changes

None

Other Changes

None

Alternatives

None

Implementation

Assignee(s)

None

Milestones

None

Work Items

The API changes mentioned above have essentially been completed. Examples of potential future changes include: * Restrictions on wildcard search (currently wildcard searching is unrestricted) * Adding more attributes that can be filtered

Dependencies

None

Secondary Zones

Wed, 03 Jun 2020 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

Outline support for Secondary Zones concept in Designate.

Terminology

Term	Meaning
axfr	Full Zone Update (AXFR)
zone	Seconday Zone - Zone that transfers it’s state / data from a given set of masters

Problem description

We want to support Secondary Zones in order to lay the grounds for higher level features like incoming (A|I)XFR, Federation and others (will be separate specs).

Proposed change

We introduce a new concept of a Secondary zone within Designate that has external Masters. In term this means that Designate will be acting as a Slave.

In MiniDNS we add support for inbound AXFR and NOTIFY.

In terms of Pools a Secondary Zone would be placed equally as a Primary

API Changes

API endpoints for recordsets should disallow any Create, Update, Delete action.

status fields will be the same as with Primary Zones.

New Zone parameters

Parameter	Type	Required	Default	Description
type	ENUM(PRIMARY, SECONDARY)	No	false	Primary or Secondary
masters	List	No	None	Zone Masters
transferred_at	Datetime		None	Last successful transfer

Create Secondary Zone

When creating a new zone the managed_resource_email is used as the initial email.

The fields version will be 1 since it’s not yet transferred and transferred_at as null.

POST /v2/zones HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json
Content-Type: application/json

{
  "zone": {
    "name": "example.org.",
    "description": "This is an example zone.",
    "masters": ["10.0.0.1", "10.0.0.2:5354"],
    "type": "SECONDARY",
  }
}

HTTP/1.1 201 Created
Content-Type: application/json

{
  "zone": {
    "id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "pool_id": "572ba08c-d929-4c70-8e42-03824bb24ca2",
    "project_id": "4335d1f0-f793-11e2-b778-0800200c9a66",
    "name": "example.org.",
    "email": "managed@foo.co",
    "serial": 1404757531,
    "status": "ACTIVE",
    "description": "This is an example zone.",
    "masters": ["10.0.0.1", "10.0.0.2:5354"],
    "type": "SECONDARY",
    "transferred_at": null,
    "version": 1,
    "created_at": "2014-07-07T18:25:31.275934",
    "updated_at": null,
    "links": {
      "self": "https://127.0.0.1:9001/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3"
    }
  }
}

Get Secondary Zone

Retrieves a secondary zone with the specified ID.

Example of GET on a untransferred zone:

GET /v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3 HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json
Content-Type: application/json

HTTP/1.1 200 OK
Vary: Accept
Content-Type: application/json

{
  "zone": {
    "id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "pool_id": "572ba08c-d929-4c70-8e42-03824bb24ca2",
    "project_id": "4335d1f0-f793-11e2-b778-0800200c9a66",
    "name": "example.org.",
    "email": "managed@foo.co",
    "serial": 1404757531,
    "status": "ACTIVE",
    "description": "This is an example zone.",
    "masters": ["10.0.0.1", "10.0.0.2:5354"],
    "type": "SECONDARY",
    "transferred_at": null,
    "version": 1,
    "created_at": "2014-07-07T18:25:31.275934",
    "updated_at": null,
    "links": {
      "self": "https://127.0.0.1:9001/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3"
    }
  }
}

List Secondary Zones

To filter on zone type do type=<PRIMARY|SECONDARY> as query parameters.

Below there is examples of a Zone that’s not transferred yet and one that is.

GET /v2/zones?type=SECONDARY HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json
Content-Type: application/json

HTTP/1.1 200 OK
Vary: Accept
Content-Type: application/json

{
  "zones": [{
    "id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "pool_id": "572ba08c-d929-4c70-8e42-03824bb24ca2",
    "project_id": "4335d1f0-f793-11e2-b778-0800200c9a66",
    "name": "example.org.",
    "email": "managed@foo.co",
    "serial": 2014120100,
    "status": "ACTIVE",
    "description": "This is an example zone.",
    "masters": ["10.0.0.1", "10.0.0.2:5354"],
    "type": "SECONDARY",
    "transferred_at": "2014-07-07T18:25:31.275934",
    "version": 2,
    "created_at": "2014-07-07T18:25:31.275934",
    "updated_at": "2014-07-07T18:25:31.275934",
    "links": {
      "self": "https://127.0.0.1:9001/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3"
    }
  }, {
    "id": "fdd7b0dc-52a3-491e-829f-41d18e1d3ada",
    "pool_id": "572ba08c-d929-4c70-8e42-03824bb24ca2",
    "project_id": "4335d1f0-f793-11e2-b778-0800200c9a66",
    "name": "example.net.",
    "email": "managed@foo.co",
    "serial": 1404756682,
    "status": "ACTIVE",
    "description": "This is another example zone.",
    "masters": ["10.0.0.1", "10.0.0.2:5354"],
    "type": "SECONDARY",
    "transferred_at": null,
    "version": 1,
    "created_at": "2014-07-07T18:22:08.287743",
    "updated_at": null,
    "links": {
      "self": "https://127.0.0.1:9001/v2/zones/fdd7b0dc-52a3-491e-829f-41d18e1d3ada"
    }
  }],
  "links": {
    "self": "https://127.0.0.1:9001/v2/zones"
  }
}

Update a Secondary Zone

Changes the specified attribute(s) for an existing zone.

In the example below, we update one of the masters to 10.0.0.3.

NOTE: In terms of a Secondary Zone only the following fields below are editable.

Parameter	Description
description	Description of Zone
masters	Master servers

PATCH /v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3 HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json
Content-Type: application/json

{
  "zone": {
    "masters": ["10.0.0.1", 10.0.0.3:1053"]
  }
}

HTTP/1.1 200 OK
Content-Type: application/json

{
  "zone": {
    "id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "pool_id": "572ba08c-d929-4c70-8e42-03824bb24ca2",
    "project_id": "4335d1f0-f793-11e2-b778-0800200c9a66",
    "name": "example.org.",
    "serial": 0,
    "status": "ACTIVE",
    "description": "This is an example zone.",
    "masters": ["10.0.0.1", "10.0.0.3:1053"],
    "type": "SECONDARY",
    "transferred_at": "2014-07-07T18:25:31.275934",
    "version": 1,
    "created_at": "2014-07-07T18:25:31.275934",
    "updated_at": null,
    "links": {
      "self": "https://127.0.0.1:9001/v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3"
    }
  }
}

Delete Secondary Zone

DELETE /v2/zones/a86dba58-0043-4cc6-a1bb-69d5e86f3ca3 HTTP/1.1
Host: 127.0.0.1:9001

HTTP/1.1 204 No Content

Central Changes

Disallow changing of a zone from PRIMARY <> SECONDARY.

Add a periodic task that loops over secondary zones looking at their transferred_at and does a call to MDNS to trigger a new AXFR to keep the zone updated.

Storage Changes

Modify Table - domains

Column	Type	Nullable?	Unique?	Notes	Action
type	EMUM(PRIMARY, SECONDARY)	No	No	Zone type	add
transferred_at	DATETIME	Yes (Not transferred)	No	Last transfer at	add

New Table - domain_attributes

A new table to store any metadata / attributes that doesn’t need to be on the on the domains table.

An index across domain_id, key, value.

Column	Type	Nullable?	Unique?	Notes	Action
id	UUID	no	Yes	ID for this attribute	add
domain_id	FK to Domain UUID	no	No	Domain ID this attribute belongs to	add
key	ENUM(masters)	no	No	Zone type	add
value	VARCHAR	no	No	Master servers for Zone	add

MiniDNS Changes

Zone Creation

When a zone is created currently a notification is sent to mdns, we’ll plugin here and do a AXFR if zone.type is SECONDARY.

NOTIFY

We need to change __call__ to pass NOTIFY down to self._handle_notify().

Receives a NOTIFY
Query the SOURCE of the NOTIFY for SOA
Compare response serial vs local if it doesn’t match then continue to next step.
Do a AXFR towards the server that sent the NOTIFY.
Call dnsutils.from_dnspython to get Domain a’la Designate version
Call Central to method with the data from #5 to update the domain.

New - RequestHandler._handle_notify(context, request)

Handles a Notification and eventually hands off to do a AXFR.

Parameter	Description	Required
context	Security context information.	Yes
request	The DNS request	Yes

New - Service.zone_sync(context, zone, master_addr=None)

A method utilized by any method in MDNS that needs to do a AXFR.

Parameter	Description	Required
context	Security context information.	Yes
zone	A objects.Domain object	Yes
master_addr	Address to use for the AXFR	No

Future work

The below is out of scope for this.

A task for /zones/<id>/tasks or so should be added in the future to allow a “forced” AXFR via the API.
A task to switch from a SECONDARY > PRIMARY.

Implementation

Assignee(s)

Primary assignee:: <https://launchpad.net/~endre-karlson>

Milestones

Target Milestone for completion:: Kilo-1

Work Items

Add new columns to storage
Extend Zones API to allow CRUD for Secondary Zones
Extend Central logic to check if it’s a Master / Secondary Zone.

Dependencies

Zone Import Refactor

Server Pools API

Wed, 03 Jun 2020 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/server-pools-api

This spec outlines the API changes needed for Server Pools. It is part of a larger group of specs describing the design of Designate Server Pools. The main blueprint is located here.

Problem description

Server Pools will need to be created, read, updated and deleted. An API needs to exist in order to carry out these operations.

Proposed change

This spec defines in detail the API changes needed for Server Pools.

API Changes

API Details: Create/List/Patch/Delete Pools

Tip

These should all be restricted to admins for general usage.

Verb	Resource	Description
GET	/pools	Returns all pools
GET	/pools/{pool_id}	Returns a specific pool with additional details
POST	/pools	Creates a new pool
PATCH	/pools/{pool_id}	Updates the pool with data in the request
DELETE	/pools/{pool_id}	Deletes the pool

GET /v2/pools

When no id is specified, all pools are returned. No body is provided in the request.

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Location: /v2/pools

{
    "pools": [
        {
            "name": "US Pool 1",
            "id": "cf1fafc0-f2f0-11e3-ac10-0800200c9a66",
            "links": {
                "self" : "/v2/pools/cf1fafc0-f2f0-11e3-ac10-0800200c9a66"
            },
            "created_at": "2014-09-09T19:34:21.819615",
            "updated_at": null
        },
        {
            "name": "US Pool 2",
            "id": "cddda8f0-f558-11e3-a3ac-0800200c9a66",
            "links": {
                "self" : "/v2/pools/cddda8f0-f558-11e3-a3ac-0800200c9a66"
            },
            "created_at": "2014-09-09T19:34:21.819615",
            "updated_at": null
        }
    ],
    "links": {
        "self": "/v2/pools"
    }
}

GET /v2/pools/{pool_id}

When an id is provided in the url, the specific pool is returned with more detail. No body is provided in the request.

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Location: /v2/pools/cf1fafc0-f2f0-11e3-ac10-0800200c9a66

{
    "pool": {
        "name": "US Pool 1",
        "id": "cf1fafc0-f2f0-11e3-ac10-0800200c9a66",
        "attributes": {
            "geo-ip": true,
            "anycast": false,
            "scope": public,
        },
        "pool_servers": [
            "1.2.3.4:53",
            "5.4.3.2:53"
        ],
        "nameservers": [
            {"priority": 1, "hostname": "ns1.example.com."},
            {"priority": 2, "hostname": "ns2.example.com."}
        ],
        "project_id": "<uuid>",
        "provisioner": "unmanaged",
        "created_at": "2014-09-04T19:34:20.819723",
        "updated_at": null
    },
    "links": {
        "self": "/v2/pools/cf1fafc0-f2f0-11e3-ac10-0800200c9a66"
    }
}

POST /v2/pools/

When a new Pool is created, the user must supply the name and scope. Initially, only “public” scope is supported. The other values shown here are optional. If the name is the same as an existing Pool, the return code will be 409 Conflict.

Request

HTTP/1.1
Content-Type: application/json; charset=UTF-8
Location: /v2/pools/

{
    "pool": {
        "name": "US Pool 3",
        "attributes": {
            "scope": public
        },
        "pool_servers": [
            "10.11.12.13:53",
            "13.12.11.10:53"
        ],
        "nameservers": [
            {"priority": 1, "hostname": "ns1.example.com."}
            {"priority": 2, "hostname": "ns2.example.com."}
        ]
    }
}

Response

HTTP/1.1 201 Created
Content-Type: application/json; charset=UTF-8
Location: /v2/pools/

{
    "pool": {
        "name": "US Pool 3",
        "id": "cf1fafc0-f2f0-11e3-ac10-0800200c81a5",
        "attributes": {
            "scope": public,
        }
        "pool_servers": [
            "10.11.12.13:53",
            "13.12.11.10:53"
        ],
        "nameservers": [
            {"priority": 1, "hostname": "ns1.example.com."},
            {"priority": 2, "hostname": "ns2.example.com."}
        ],
        "created_at": "2014-09-04T19:34:20.819723",
        "updated_at": null
    },
    "links": {
        "self": "/v2/pools/cf1fafc0-f2f0-11e3-ac10-0800200c81a5"
    }
}

PATCH /v2/pools/{pool_id}

To modify a Pool, a PATCH request is submitted. If it is successful, a 200 is returned.

Request

HTTP/1.1
Content-Type: application/json; charset=UTF-8
Location: /v2/pools/cf1fafc0-f2f0-11e3-ac10-0800200c81a5

{
    "pool": {
        "pool_servers": [
            "4.5.6.7:53"
        ]
    }
}

Response

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Location: /v2/pools/cf1fafc0-f2f0-11e3-ac10-0800200c81a5

{
    "pool": {
        "name": "US Pool 3",
        "id": "cf1fafc0-f2f0-11e3-ac10-0800200c81a5",
        "attributes": {
            "scope": public,
        },
        "pool_servers": [
            "10.11.12.13:53",
            "13.12.11.10:53",
            "4.5.6.7:53"
        ],
        "nameservers": [
            {"priority": 1, "hostname": "ns1.example.com."},
            {"priority": 2, "hostname": "ns2.example.com."}
        ],
        "created_at": "2014-09-04T19:34:20.819723",
        "updated_at": "2014-09-10T19:33:10.819555"
    },
    "links": {
        "self": "/v2/pools/cf1fafc0-f2f0-11e3-ac10-0800200c81a5"
    }
}

DELETE /v2/pools/{pool_id}

When deleting a Pool, the user must supply the id in the url. The request body and return body are empty. A 204 is returned

HTTP/1.1 204 No Content Content-Type: application/json; charset=UTF-8 Location: /v2/pools/cf1fafc0-f2f0-11e3-ac10-0800200c81a5

Central Changes

The pool calls will have to be added to central to allow the CRUD of Server Pools.

Storage Changes

The pool calls will have to be added to storage. The new tables are described in a different spec.

Implementation

Assignee(s)

Primary assignee:: https://launchpad.net/~betsy-luzader

Milestones

Target Milestone for completion:: Kilo

Work Items

Create pools controller and view
Add the calls to Central
Add the calls to Storage
Write tests

Dependencies

Server Pools Storage: https://review.openstack.org/#/c/113447/
Server Pools Service: https://review.openstack.org/#/c/113462/
Server Pools MiniDNS Support: https://review.openstack.org/#/c/112688/

Designate Split View

Mon, 01 Jun 2020 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/split-view https://bugs.launchpad.net/designate/+bug/1875939

Split view is an important feature most DNS servers have support for based on the need to split internal, external view of the IPs and hostnames, most of the companies have this feature on their existing DNS structers and adding support for it Designate will make it easier for the user to integrate Designate into the current user systems

The below links provide insight on the split view feature and use cases: * http://techgenix.com/you_need_to_create_a_split_dns/ * https://en.wikipedia.org/wiki/Split-horizon_DNS

Problem description

The current implementation of Designate does not support the split view, users that want to implement this must either patch Designate code or somehow do it in the backend of Designate (bind, powerdns, ….)

Proposed change

Introduce a new type of zone (split_view) in Designate. when a zone with that type is created Designate will do the following:

Create the same zone twice in different views (internal, external), this will require the pool.yaml to be configured with two pools seperated with attributes internal/external Note: the user may use view parameter in bind or define two instances of powerdns to acheive that
Create two TSIG keys one for each zone and attach the TSIG to the right zone
Provide AXFR based on TSIG key so if the TSIG sent is exteranl give the external view the user should provide a regex for external/internal view that match the IPs that should be included in each view, so if the IP match internal regex specified by user and the AXFR request is signed with internal TSIG the IP will be included in zone AXFR response
Also notify requests should be signed with the right TSIG (internal, external) the user should provide a regex for internal/external that match the IPs that should be Included/Excluded from each view and based on that the TSIG key can be selected

Backend should be configured as follows:

All AXFR request should be signed with TSIG, so the AXFR can return the right view
Most of Designate backend agents listed in the link below https://docs.openstack.org/designate/pike/contributor/backends/index.html have support for TSIG AXFR/split_view (initial development could be for bind, powerdns)

[bind] https://www.slashroot.in/how-to-configure-split-horizon-dns-in-bind https://bind9.readthedocs.io/en/latest/advanced.html#instructing-the-server-to-use-a-key [powerdns] https://www.frank.be/implementing-bind-views-with-powerdns/ https://doc.powerdns.com/authoritative/tsig.html#provisioning-signed-notification-and-axfr-requests [djbdns] djbdns does not support TSIG based on the link below https://en.wikipedia.org/wiki/Djbdns https://www.fefe.de/djbdns/#splithorizon [gdnsd] gdnsd not sure if it support split view/tsig AXFR [infoblox] https://docs.infoblox.com/display/NAG8/Chapter+18+DNS+Views https://docs.infoblox.com/display/NAG8/Enabling+Zone+Transfers [knote] https://www.knot-dns.cz/docs/2.6/html/configuration.html https://knot-resolver.readthedocs.io/en/v2.2.0/modules.html#views-and-acls [MSDNS] https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-gssa/c0c6ffdd-a094-40b1-bbb9-bc4e5a58804f https://docs.microsoft.com/en-us/windows-server/networking/dns/deploy/split-brain-dns-deployment

Storage Changes

The Designate zones table should be updated to accept the new zone type split_view type enum(‘PRIMARY’,’SECONDARY’) to type enum(‘PRIMARY’,’SECONDARY’,’SPLIT_VIEW’)

Assignee(s)

hamza alqtaishat

Server Pools Manager

Mon, 07 Jan 2019 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/server-pools-service

This specification outlines the Pool Manager, Central, backend driver, and storage changes needed to support the new Pool Manager service.

Problem description

Coordinating DNS operations across many different backends is difficult, especially when there is a great number of DNS servers. A Pool Manager service is needed to manage the changes from the Designate database to the many DNS servers. A Pool Manager will also track the status of those changes. When this specification is implemented, a Pool Manager will be used to manage a pool with multiple DNS servers, even if those DNS servers are of different types.

Proposed change

API Changes

None

Pool Manager Changes

A new Designate service, called designate-pool-manager, will be created. This is the Pool Manager. The Pool Manager will get its configuration from the configuration file when it is instantiated.

The configuration section is called [service:pool_manager]. The options for this section are:

Parameter	Default	Required	Notes
pool_name	‘default’	Yes	The pool name of the pool managed by this instance of the Pool Manager
threshold_percentage	100	Yes	The percentage of servers requiring a successful update for a domain change to be considered active
poll_timeout	30	Yes	The time to wait for a NOTIFY response from a name server
poll_retry_interval	2	Yes	The time between retrying to send a NOTIFY request and waiting for a NOTIFY response
poll_max_retries	3	Yes	The maximum number of times minidns will retry sending a NOTIFY request and wait for a NOTIFY response
periodic_sync_interval	120	Yes	The time between sychronizing the servers with Storage

The Pool Manager will contain a map of the servers to instantiated backend drivers. The backend driver will not be responsible for reading the configuration information as the Pool Manager will read the global backend driver and server specific backend driver sections from the configuration file and pass the backend driver configuration to the backend driver for instantiation. This map will be created when the Pool Manager is instantiated. Please refer to the Backend Driver Changes section in the Storage Pools - Storage specification for more information concerning the global backend driver and server specific backend driver sections.

The methods in the base class for the Pool Manager service include:

create_domain(context, domain)

Parameter	Description	Required
context	Security context information.	Yes
domain	The designate domain object.	Yes

Return Value

No return value.

Design Considerations

Loop through each server in the pool and call the backend driver to create the domain. For each call to the backend driver, the status is stored in the pool_manager_status table with an action of ‘CREATE’ and a second row is created with an action of ‘UPDATE’. Successful creations have a status of ‘SUCCESS’ and failed creations have a status of ‘ERROR’. The ‘UPDATE’ action row has no initial status. Check to see if a consensus exists using the pool_manager_status table. Consensus exists if the number of servers for the domain with a successful creation exceed the threshold_percentage. If consensus exists, the Central update_status method is called using the serial number used when creating the domain and a status of ‘SUCCESS’. If consensus does not exist, the Central update_status method is called using the serial number used when creating the domain and a status of ‘ERROR’.

Cast vs. Call

This is an RPC cast. Communication about the status of the domain creation will be handled using the Central update_status method.

delete_domain(context, domain)

Parameter	Description	Required
context	Security context information.	Yes
domain	The designate domain object.	Yes

Return Value

No return value.

Design Considerations

Loop through each server in the pool and call the backend driver to delete the domain. For each call to the backend driver, the status is stored in the pool_manager_status table with an action of ‘DELETE’. Successful deletions have a status of ‘SUCCESS’ and failed deletions have a status of ‘ERROR’. Check to see if a consensus exists using the pool_manager_status table. Consensus exists if the number of servers for the domain with a successful deletion exceed the threshold_percentage. If consensus exists, the Central update_status method is called using the serial number used when deleting the domain and a status of ‘SUCCESS’. If consensus does not exist, the Central update_status method is called using the serial number used when creating the domain and a status of ‘ERROR’.

Cast vs. Call

This is an RPC cast. Communication about the status of the domain deletion will be handled using the Central update_status method.

update_domain(context, domain)

Parameter	Description	Required
context	Security context information.	Yes
domain	The designate domain object.	Yes

Return Value

No return value.

Design Considerations

Loop through each server in the pool and call the minidns notify_zone_changed method. Loop through each server again and call the minidns poll_for_serial_number method.

Cast vs. Call

This is an RPC cast. Communication about the status of the domain update will be handled using the Central update_status method which is called by the Pool Manager update_status method. The minidns poll_for_serial_number method invokes the Pool Manager update_status method when it completes.

update_status(context, domain, name_server, status, serial_number)

Parameter	Description	Required
context	Security context information.	Yes
domain	The designate domain object.	Yes
name_server	The name server for which this serial number is applicable.	Yes
status	The status, ‘SUCCESS’ or ‘ERROR’.	Yes
serial_number	The serial number received from the name server for the domain.	Yes

Return Value

No return value.

Design Considerations

Reads the existing serial number from the pool_manager_status table for the server and domain. If the new serial number > the existing serial number, update the row and check to see if a consensus exists using the pool_manager_status table. Consensus exists if the number of servers for the domain with a serial number > the existing serial number exceed the threshold_percentage. Servers are discounted from participating in the consensus starting with the servers with the lowest serial numbers until the minimum number of servers needed to achieve consensus based on the threshold_percentage is realized. If the existing serial number < all the serial numbers for the remaining servers, the Central update_status method is called using the lowest (consensus) serial number for those remaining servers and a status of ‘SUCCESS’.

If > 100 - threshold_percentage servers for the domain have a status of ‘ERROR’, the Central update_status method is called using the lowest serial number greater than the consensus serial number (calculated above) and a status of ‘ERROR’.

Cast vs. Call

This is an RPC cast.

periodic_sync()

Return Value

No return value.

Design Considerations

This method is a thread that is created when Pool Manager is instantiated. The intent of this thread is to read the pool_manager_status table and perform failed create, delete, and updates operations. Additionally, the thread will call the minidns poll_for_serial_number method for each domain and server to ensure the server is synchronized with Storage.

Every period_sync_interval, this thread will perform the following operations:

Read the pool_manager_status table looking for ‘CREATE’ actions that have a status of ‘ERROR’ grouping by domains and ordering by the row create time. Check to see if a consensus already exists for the domain creation. Loop through each servers with a failed creation, using the backend driver to attempt creation. If consensus does not already exist, check for consensus and call the Central update_status if consensus is achieved.

Read the pool_manager_status table looking for ‘DELETE’ actions that have a status of ‘ERROR’ grouping by domains and ordering by the row create time. Check to see if a consensus already exists for the domain deletion. Loop through each servers with a failed deletion, using the backend driver to attempt deletion. If consensus does not already exist, check for consensus and call the Central update_status if consensus is achieved.

For each domain in the pool, read the domain’s serial number from Storage. Loop through each server in the pool and read the pool_manager_status table looking for ‘UPDATE’ actions for the domain that have a serial number < the domain’s serial number and call the minidns notify_zone_changed method.

Finally, for each domain in the pool, read the domain’s serial number from Storage. Loop through each server in the pool and call the minidns poll_for_serial_number method.

Central Changes

The Central service will be updated to use the Pool Manager instead of the backend driver. Additionally, the default_pool_name option will be removed from the [service:central] section of the Designate configuration.

All domains will be ‘PENDING’ status initially and calls to the Central update_status method by the Pool Manager will change the status.

When creating, updating, or deleting records, records will have the serial number field set to the new serial number of the domain. The task will be ‘ADD’, ‘DELETE’, or ‘UPDATE’ corresponding to the operation. The status will be ‘PENDING’.

Valid record states are:

task	status
‘ADD’	‘PENDING’
‘ADD’	‘ERROR’
‘DELETE’	‘PENDING’
‘DELETE’	‘ERROR’
‘UPDATE’	‘PENDING’
‘UPDATE’	‘ERROR’
‘NONE’	‘ACTIVE’
‘NONE’	‘DELETED’

Affected code in the Central service will be updated appropriately to align with these states.

The new method needed to update the status of domains and records is:

update_status(context, domain, status, serial_number)

Parameter	Description	Required
context	Security context information.	Yes
domain	The designate domain object.	Yes
status	The status, ‘SUCCESS’ or ‘ERROR’.	Yes
serial_number	The consensus serial number for the domain.	Yes

Return Value

No return value.

Design Considerations

If the status is ‘SUCCESS’:

Check the status of the domain and if it has a status of ‘PENDING’ or ‘ERROR’, set the status to ‘ACTIVE’.

Check the status of records for the domain. If they have a task of ‘ADD’ or ‘UPDATE’ and a status of ‘PENDING’ or ‘ERROR’, set the task to ‘NONE’ and the status to ‘ACTIVE’ if the consensus serial number >= serial number field.

Check the status of records for the domain. If they have a task of ‘DELETE’ and a status of ‘PENDING’ or ‘ERROR’, set the task to ‘NONE’ and the status to ‘DELETED’ if the consensus serial number >= serial number field.

If the status is ‘ERROR’:

Check the status of the domain and if it has a status of ‘PENDING’, set the status to ‘ERROR’.

Check the status of records for the domain. If they have a status of ‘PENDING’, set the status to ‘ERROR’ if the consensus serial number >= serial number field.

Cast vs. Call

This is an RPC call.

Backend Driver Changes

The backend driver will now be instantiated with information provided by the Pool Manager as explained in the Pool Manager Changes section. This is necessary because of server specific backend driver configurations.

The backend driver will continue to support the same configuration options they currently do, only the section names will change by adding a wildcard qualifier for the server. For example, the backend driver section for PowerDNS will now be [backend:powerdns:*]. This syntax will denote the global configuration for the backend driver. This is done to allow for server specific backend driver configurations.

The new server specific backend driver section in the configuration will be [backend:powerdns:<uuid>] where uuid is a universally unique identifier.

The options for this section are:

Parameter	Default	Required	Notes
host	None	Yes	The host name or IP address of the DNS server
port	53	Yes	The port of the DNS server
tsig_key	None	Yes	The TSIG key for the DNS server

In addition to the above options, the server specific backend driver section will support the same options as the backend driver global section. If those options are not included in the server specific backend driver section, the server configuration will default to using the global configuration option. These server specific backend driver sections will support different backends in the same pool.

The server object will be implemented. The server object encapsulates the server specific backend driver section in the configuration.

The following methods will not be used in the backend driver:

create_tsigkey(tsigkey)

update_tsigkey(tsigkey)

delete_tsigkey(tsigkey)

This is due to the only provisioner supported initially being the ‘unmanaged’ provisioner. Those methods will be used for future provisioners.

Storage Changes

A new table for the Pool Manager status will be needed. Additionally, the domains and records tables will be modified to support pools. Domains and records will be ‘PENDING’ status initially. A new status ‘ERROR’ will be possible for domains and records. Finally, a record can also be ‘DELETE_PENDING’ and ‘DELETE_ERROR’.

New Table - pool_manager_status

Column	Type	Nullable?	Unique?	Notes
id	CHAR(32)	No	Yes	PK
updated_at	DATETIME	No	No	UTC time of last update
server_id	VARCHAR(32)	No	No	Server ID
domain_id	CHAR(32)	No	No	FK to ID on domains table
status	‘SUCCESS’,’ERROR’	Yes	No	Status
serial_number	INT(11)	No	No	Serial number at time of status
action	‘CREATE’,’DELETE’,’UPDATE’	No	No	Action resulting in status

Modify Table - domains

Column	Type	Nullable?	Unique?	Default	Notes	Action
status	‘ACTIVE’,’PENDING’,’DELETED’,’ERROR’	No	No	‘PENDING’	Record status	update

Modify Table - records

Column	Type	Nullable?	Unique?	Default	Notes	Action
serial_number	INT(11)	No	No		Used for the record status	add
task	‘ADD’,’DELETE’,’UPDATE’,’NONE’	No	No	‘ADD’	Record operation task	add
status	‘ACTIVE’,’PENDING’,’DELETED’,’ERROR’	No	No	‘PENDING’	Record status	update

Other Changes

None

Alternatives

None

Implementation

Assignee(s)

Primary assignee:: https://launchpad.net/~rjrjr
Additional assignee:: https://launchpad.net/~darshan104

Milestones

Target Milestone for completion:: Kilo-1

Work Items

Pool Manager changes
Central changes
Backend driver changes
Storage changes

Dependencies

This specification relies on the Server Pools - Storage specification. This specification relies on the Server Pools - MiniDNS Support specification.

Centralize Validation Logic

Sat, 24 Nov 2018 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/validation-cleanup

Problem description

Today, validations are duplicated between the V1 and V2 APIs, and validations will be required in additional places going forward (Inbound AXFR, DynamicDNS etc). Centralizing these validations into the Designate Objects provides a single re-usable home for all entry points to use.

Proposed change

Centralizing this logic will require a number of phases to complete:

Implement an Object Registry
Implement Object Validation
Implement an “Adaptor” layer, replacing the V2 APIs Views
Migrate schemas from designate/resources/schemas into the Objects
Update the API layer (both V1 and V2) to use the new Validations and Adaptors

Object Registry

The object registry will allow for looking up a reference to any DesignateObject’s class via the class name. This will allow an object’s schema to reference other object easily.

Note

The Object Registry will not replace the standard and existing method of retrieving an Object class by importing it. The Registry provides an alternative method of obtaining a class reference using only the class name. This is useful for scenarios where we need to reference an Object outside of python code. For example, in a JSON-Schema, or in JSON messages passed from service to service with oslo.messaging.

To implement the registry, the DesignateObjectMetaclass class will be updated to track a reference to each of the Object classes as they are constructed. These references will be stored in a dictionary attached to the DesignateObject base class.

Note

The DesignateObjectMetaclass code is executed while the object classes are constructed, rather than when the object instances are created. This ensures the code is only executed once upon startup of the Designate services.

Registry lookups will be performed via a new DesignateObject.obj_cls_from_name() method, which will accept a single string argument for the Object Name.

class DesignateObject(object):
    @classmethod
    def obj_cls_from_name(cls, name):
        pass

An example usage of the registry:

class RecordSet(DesignateObject):
    FIELDS = {
        'id': {},
        'name': {},
    }

RecordSet = DesignateObject.obj_cls_from_name('RecordSet')

my_recordset = RecordSet(id='12345', name='example.org.')

Object Validation

Object Validation rules will continue to use JSON-Schema, implemented on a per-field level:

class ValidatableObject(DesignateObject):
    FIELDS = {
        'id': {
            'required': True,
            'schema': {
                'type': 'string',
                'format': 'uuid'
            }
        },
        'ttl': {
            'schema': {
                'type': 'integer',
                'minimum': 0,
                'maximum': 100
            }
        },
        'recursive': {
            'schema': {
                '$ref': 'obj://ValidatableObject/#',
            }
        },
        'nested': {
            'schema': {
                '$ref': 'obj://AnotherObject/#',
            }
        }
    }

To construct the final and complete scheme, and instanciate the schema validator, the DesignateObjectMetaclass class will be updated to call a make_class_validator(cls) method, implemented similarily to the make_class_properties(cls) method.

This make_class_validator method will assemble the per-field schema fragments into a full JSON Schema, with the necessary boilerplate being generated. Additionally, this method will construct the python-jsonschema Validator instance and attach it to the objects class as cls._obj_validator.

Finally, three new methods will be added to the base DesignateObject class:

A obj_get_schema(cls) method:

class DesignateObject(object):
    @classmethod
    def obj_get_schema(cls):
        """Returns the JSON Schema for this Object."""

A is_valid(self) method:

class DesignateObject(object):
    def is_valid(self):
        """Returns True if the Object is valid."""

A validate(self) method:

class DesignateObject(object):
    def validate(self):
        """
        Raises an InvalidObject exception if the Object is invalid

        Attached to the `errors` attribute of exception will be a
        `ValidationErrorList` object containing the details of the
        failures.
        """

An example usage of the validation:

class RecordSet(DesignateObject):
    FIELDS = {
        'id': {
            'required': True,
            'schema': {
                'type': 'string',
                'format': 'uuid'
            }
        },
        'ttl': {
            'schema': {
                'type': 'integer',
                'minimum': 0,
                'maximum': 100
            }
        }
    }

my_recordset = RecordSet(id='12345', ttl=50)

# Returns False, as the 12345 is NOT a UUID.
my_recordset.is_valid()

try:
    # Raises an InvalidObject exception, as the 12345 is NOT a UUID.
    my_recordset.validate()
except InvalidObject as e:
    LOG.warning('Invalid Object, Errors below:')

    for error in e.errors:
        LOG.warning('Error at path %s, Message: %s', e.absolute_path,
                    e.message)

Object Adaptors

Object Adaptors will replace the current V2 API Views, allowing for a structured way to convert from Object to V1 or V2 API formats. This will include renaming of fields in standard output, rendered JSON Schemas, as well as in ValidationError messages, and will support hiding fields which should not be visible in the matching API version.

Note

Below is a WIP mockup - Expect changes!

Example usage of the Object Adaptors:

# Standard Object Definition
class Domain(DesignateObject):
    FIELDS = {
        'id': {
            'required': True,
            'schema': {
                'type': 'string',
                'format': 'uuid'
            }
        },
        'name': {
            'schema': {
                'type': 'string',
                'pattern': 'domainname'
            }
        },
        'ttl': {
            'schema': {
                'type': 'integer',
                'minimum': 0,
                'maximum': 100
            }
        },
        'version': {
            'schema': {
                'type': 'integer',
                'minimum': 0,
                'maximum': 100
            }
        }
    }


# Define the V2 API Adaptor for the Domain Object above
class DomainAdaptorV2(DesignateObjectAdaptorV2):
    obj_cls = Domain
    obj_list_cls = DomainList

    # Any fields NOT specificed will not be returned by the API.
    FIELDS = {
        'id': {
            # No V2 Specific Customization Needed
        }
        'ttl': {
            # Let's rename "ttl" to "default_ttl" in V2
            'name': 'default_ttl'
        }
    }


# Use the Adaptor in the API
class ZonesController(rest.RestController):
    _adaptor = DomainAdaptorV2()

    @pecan.expose(template='json:', content_type='application/json')
    @utils.validate_uuid('zone_id')
    def get_one(self, zone_id):
        """Get Zone"""

        # Real life would Fetch a zone from designate-central
        domain = Domain(id='2b9e1b86-d4f1-42d2-88ff-b888f2dd068a'
                        name='example.com.',
                        ttl=50)

        return self._adaptor.render(domain, single=True)

    @pecan.expose(template='json:', content_type='application/json')
    def post_all(self):
        """Create Zone"""
        request = pecan.request
        response = pecan.response
        context = request.environ['context']

        # The Adaptor class will parse the incoming JSON into an
        # approperiate Object instance, trigger validation, and raise
        # an exception if there are any failures. The
        # `FaultWrapperMiddleware` will catch and render this exception.
        domain = self._adaptor.parse(request.body_dict, single=True)

        # Create the Domain
        domain = self.central_api.create_domain(context, domain)

        # Prepare the response headers and status
        response.status_int = 201
        response.headers['Location'] = '<url for new zone>'

        # Send the response
        return self._adaptor.render(domain)

Other Changes

Any other changes to Designate, broken down by which sub system is being changed

Implementation

Assignee(s)

Primary assignee:: kiall

Milestones

Target Milestone for completion:: Kilo-1

Work Items

Work items are as per the Proposed change section.

Dependencies

No known dependencies

Zone Exists Event

Sat, 24 Nov 2018 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/zone-exists-event

Problem description

Designate emits several events for use by Ceilometer for Metering & Billing purposes. Currently, these events are sent for zone/rrset/etc create, update and delete actions. As these events are emitted over RabbitMQ, we cannot guarantee delivery. In order to ensure accurate billing for customers, we need to provide a mechanism for identifying “lost” events. This is particularity important for Zone create/delete events, as per-Zone/per-Timeperiod is the most common billing unit.

The accepted standard solution to the loss of create/delete events is for each service to emit a periodic “exists” event, this allows Ceilometer to, for example, identify that N exists events in a row were not received, surmising that a “delete” event was missed.

In other services like Nova there is a clear “owner” for each core resource (e.g. an instance is owned by the compute node it resides on), this allows each nova-compute instance to periodically emit a compute.instance.exists event for a small number of instances. Designate has no such clear owner, and many thousands of zones may belong to a single pool (the smallest grouping we currently have available). This poses a problem, in that emitting 100’s of thousands of events, or more, each hour may be too demanding for the active pool manager.

Proposed change

We will introduce a new service designate-zone-manager which will handle all periodic tasks relating to the zones it is responsible for. Initially, this will only be the periodic dns.domain.exists event, but over time we will add additional tasks, for example, polling of Secondary zones at their refresh intervals.

A concept of “zone shards” will be introduced, where every zone will be allocated to a shard based on the first three characters of the zones UUID. This will provide for 4,096 distinct shards to be distributed over the set of available designate-zone-manager processes, ensuring that no single designate-zone-manager is responsible for more zones than it can reasonably handle. With 5 million zones, each shard should contain approx 1.2k zones.

Finally, distribution of shards to available designate-zone-manager processes will be handled by the OpenStack Tooz library. Tooz provides the building blocks required to implement a “Partitioner”, using it’s Group Membership APIs to divvy up the available shards between active workers, including dynamically re-allocating based on membership changes.

API Changes

None

Central Changes

None

Storage Changes

A new column will be added to the domains, recordsets and records tables, this column will be populated by the storage driver with the integer representation of the first 3 characters of the UUID, giving a whole number with a value between 0 and 4095. We add the shard to the RecordSets/Records tables now, as we know it will be used in a future blueprint (ALIAS records).

Domains Table Additions

Row	Type	Nullable?	Unique?
shard	int(2)	No	No

RecordSets/Records Table Additions

Row	Type	Nullable?	Unique?
domain_shard	int(2)	No	No

Other Changes

A new service, designate-zone-manager will be introduced. This service will extend the Designate base Service class, but will not include the typical RPCService mixin. For this initial use case, there is no requirement for an RPC API.

This service will use the existing oslo-incubator ThreadGroup.add_timer() methods for scheduling tasks, and the tooz library for group membership.

The timer interval will be exposed as a configuration value, defaulting to 3600 seconds. The interval will be independent of any other timers introduced into this service in the future.

Group membership will be implemented as a Service mixin, similar to RPCService, allowing for it’s inclusion in other services such as designate-pool-manager.

The dns.domain.exists event format will be identical to the domain create/update/delete event format, with the only difference being the event name.

Finally, the list of Zones each designate-zone-manager is responsible for will be gathered at the start of each timer interval, in batches of a configurable size, based on the range of shard’s allocated.

Implementation

Assignee(s)

Primary assignee:: Endre Karlson

Milestones

Target Milestone for completion:: Liberty-2

Work Items

Implement “Partitioner” based on Tooz
Implement GroupMembership mixin (Name TBD)
Create new designate-zone-manager service
Implement periodic exists event

Dependencies

Requires the OpenStack Tooz library be added to our requirements
Requires infrastructure for the OpenStack Tooz library (memcache, redis, or zookeeper)

Alias Records

Tue, 13 Nov 2018 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/alias-records

ALIAS records (aka ANAME’s, root CNAME’s, root domain redirect, zone apex CNAME’s) are a relatively new “addition” to DNS. These pseudo Resource Records (RR) are used to bypass restrictions placed within the DNS protocol that forbid the presence of a CNAME RR next to any other RR. This prevents some modern usage of DNS, for example, many CDNs or other static-content hosting sites like GitHub Pages will require users to CNAME e.g. “www.example.org.” towards their frontend load balancers. A CNAME is used to ensure the provider can easily make infrastructure changes without requiring every customer to implement DNS changes in-sync with their deployment schedule.

As this is not a true feature within the DNS protocol, support for ALIAS RR’s must be implemented entirely within the end users authoritative DNS infrastructure.

Problem description

DNS RFCs do not support a CNAME record set for the root domain:

example.com IN CNAME example.github.com

As a result, this capability is not widely supported by DNS backends.

Proposed change

Designate will expose a new “ALIAS” record set type via API; the record set will be flattened on a periodic basis into an “A” or “AAAA” record set for propagation to DNS backends. This solution is backend-independent and complies with RFCs implementations.

Acceptance Criteria

An end user can create an “ALIAS” record set in the API and see that record set through all other API requests (e.g. get record sets, export zone, import zone)
All “ALIAS” records will be flattened to “A/AAAA” record sets for all backends.
An IP change to example.github.com will result in an update the IP address of the flattened “A” or “AAAA” record sets available on the backend.
Operators can configure a custom caching name sever to use for resolution and flattening of ALIAS record sets.
Operators can set the polling interval for the flattening process.

Note

Not recommended for use with CDN implementations.

API Changes

RRSets with the “visible” value (described below) set to “mdns” will be filtered from view within the API.

Additionally, ALIAS RRSets will not be visible in the V1 API. Instead, a dynamically generated TXT record will be included to indicate the existence of the ALIAS record that is only editable via the V2 API.

Example ALIAS record set creation request:

POST /v2/zones/2150b1bf-dee2-4221-9d85-11f7886fb15f/recordsets HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json
Content-Type: application/json

{
  "name" : "example.org.",
  "description" : "This is an example ALIAS record set.",
  "type" : "ALIAS",
  "ttl" : 3600,
  "records" : [
    "example.github.com."
  ]
}

Flatten Task

The process of “flattening” the ALIAS record can be triggered by the user. For example, if a user starts getting error messages the resolution isn’t working, the user can make an API request to force an update.

Example Manual Flatten Request:

POST /v2/zones/<zone_id>/recordsets/<recordset_id>/tasks/flatten HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json

The request body should be empty. Also, it is recommended the user configure rate limits on this endpoint accordingly. This rate limiting is outside the scope of this spec.

Exporting

When a zone is exported as a zone file, the export should contain two lines, the ALIAS record, commented out, and the most recent A record.

For example:

; example.com. IN ALIAS example.org.
example.com.  IN  A     192.0.2.1

This allows the user to have a valid zone file to import, while providing visibility that the A record was generated from an ALIAS record.

Central Changes

Central’s create and update RecordSet methods will be updated to call a new RPC method implemented on the Zone Manager service to trigger immediate ALIAS flattening when ALIAS RRSets are created / updated. If the resolution fails, the record will be placed in the ERROR status.

Central’s delete RecordSet method will be updated to remove the associated flattened A and AAAA RecordSets - the mechanism for identifying these related RRSets is detailed in Zone Manager Changes.

Additionally, ALIAS RRSets will be treated somewhat similarly to CNAMEs. It will be invalid to place an ALIAS RRSet next to an A or AAAA RecordSet. The recordset placement validation will be updated to handle this case.

MiniDNS Changes

MiniDNS will be updated to display RRSets with the “visible” field (described below) set to ‘mdns’ or ‘all’, ensuring no attempt is made to include non-RFC compliant RRSets within AXFRs.

Zone Manager Changes

First, for each zone it manages, designate-zone-manager service will find any ALIAS recordsets associated with the zone. The zone manager will then issue a successful A/AAAA DNS query to get the IP addresses associated with the ALIAS target. Designate’s database will be updated to the “real” values, incrementing the SOA serial to trigger an AXFR towards the public facing nameservers.

If this query fails and the ALIAS target can’t be resolved, the A/AAAA records will not be updated and the ALIAS RR will be put in STALE status.

The interval at which ALIAS records are flattened will be configurable, defaulting to 1800 seconds (30 minutes).

Finally, a new RPC method will be implemented to trigger immediate ALIAS flattening for a specific Zone, allowing for newly created ALIAS RR’s to be usable before the first periodic interval is hit. This RPC method will also be utilized by the ALIAS RRset flattening task endpoint.

Storage Changes

Recordsets: update column “type”

Add “ALIAS” to “type”.

Recordsets: New Column

Field	Type	Null	Key	Default	Extra
visible	enum(‘all’, ‘api’, ‘mdns’)	NO	MUL	‘all’

Sample data

For the example scenario described above, corresponding database values are the following (columns omitted for brevity):

Recordsets Table

id	domain_id	name	tenant_id	type	visible
1230	768	example.github.com.	391	ALIAS	‘api’
0100	768	example.org.	391	A	‘mdns’

Records Table

id	recordset_id	data	managed	managed_resource_type	managed_resource_id
3211	1230	example.org.	0	NULL	NULL
3212	0100	192.168.1.10	1	ALIAS	3211

Alternatives

Some DNS backends support flattening processes (e.g. PowerDNS). An alternative implementation is to create a new record set type called “ALIAS” that integrates with each respective backend’s implementation.

Implementation

Assignee(s)

Primary assignee:: Eric Larson <eric.larson@rackspace.com>

Milestones

Target Milestone for completion:: Mitaka-1

Work Items

Implement support for hiding RRSets in the API and MiniDNS
Implement the periodic ALIAS flattening within the designate-zone-manager service
Update Central’s CUD RecordSet methods for ALIAS support

Dependencies

None

Integrate yadifa dns backend to designate

Thu, 22 Dec 2016 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/yadifa-dns-backend

Problem description

There are users or env that may want to use YADIFA as backend which is not supported by designate as of now.

Proposed change

YADIFA is a lightweight authoritative Name Server with DNSSEC capabilities. It has a simple configuration syntax and can handle more queries per second while maintaining one of the lowest memory footprints in the industry. YADIFA also has one of the fastest zone file load times ever recorded on a name server.

Functionality:

. authoritative name server . DNS UPDATE . DNS NOTIFY . AXFR . IXFR . full featured client (yadifa), which can be used to control the server . key management, including a tool to generate dnssec keys . multi-master support . support for other network models . detect and configure hyperthreading . support for openssl 1.1.0 API

Use Cases:

Alternative for BIND and NSD for public TLD slave. - Clean Implementation - High query rate and portable. - RFC compliant: - . Authoritative - . DNSSEC support - . AXFR/IXFR (master and slave)
Dynamic updates - Dynamic updated(including continuous signing) - Generic, extensible storage backend.
Zone management - Dynamic zone provisioning.
Recursive Nameserver - Recursion - Validation

API Changes

N/A

Central Changes

N/A

Storage Changes

N/A

Other Changes

Add driver ‘impl_yadifa’ to designate.backends using v2 api . Authentication is done via the keystoneclient auth / session using the provided credentials in the backend options.

Add designate devstack plugin ‘backend-yadifa’ to configure yadifa as backend while installing designate using devstack. Also add docs related to installation and configuration of yadifa dns backend.

Alternatives

N/A

Implementation

N/A

Assignee(s)

Primary assignee:: Sonu kumar https://launchpad.net/~sonu-bhumca11
IRC Nick Name:: sonuk

Milestones

Target Milestone for completion:

Work Items

. Create YADIFA backend . Create YADIFA backend docs . Create Devstack backend plugin for YADIFA . Create experimental gate job

Naming Conventions for Records

Mon, 30 May 2016 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/different-format-for-ipv4-and-ipv6

Naming Conventions for records Right now we have a single format to define FQDN for records created through designate sink.

Problem description

At present there is only one format we can define for both IPv4 and IPv6 ips. If a user want to have different naming convention for records created through Sink service, we do not have a way to do that since there is only one option in designate.conf where we define the FQDN format.

Proposed change

In designate.conf we should give an option under the Sink configuration where we can define two different formats for IPv4 and IPv6. Similarly we need to make appropriate changes in sink code under designate/notification_handler/ for base.py, nova.py and neutron.py

The new format options in designate.conf would be “formatv4” and “formatv6”.

To help users from older versions, we will still support “format” which will work for IPv4 addresses. “format” or “formatv4” can be used interchangebaly but if both of them are defined then “formatv4” will be accepted along with a Warning message.

SINK Changes

Base.py, nova.py and neutron.py needs to be changed to get the format type from designate.conf. The new formats that we need to get are formatv4 and formatv6 which will now be implemented in designate.conf.

API Changes

None

Central Changes

None

Storage Changes

None

Other Changes

None

Alternatives

None

Implementation

Assignee(s)

Primary assignee:: Kumar Acharya<ma501v>

Milestones

Target Milestone for completion:: Mitaka-3

Work Items

Define the format for IPV4 and IPv6 FQDN and Implement it in designate.conf Code changes required in designate

Dependencies

formatv4 and formatv6 should be implemented in designate.conf as new format types

Get the Zone transfer accepts list

Tue, 24 May 2016 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/zone-transfer-accept-list

Currently listing of all the accepted zone transfer is not supported by the designate API.

This bp adds the support request that will enable users to view the list of all the accepted zone transfers.

Problem description

A Zone Transfer is the term used to refer to the transferring of ownership of a zone from one tenant say A over to another tenant B.

The /zones/tasks/transfer_accepts API provides a way to list all the zone transfer accepts that will enable users to view all the accepted zone ownership transfers.

Proposed change

API Changes

Expose the list of accepted zones transfer requests in the v2 API as “/zones/tasks/transfer_accepts”.

GET /v2/zones/tasks/transfer_accepts

This generates the list of accepted zone transfer requests

Example Request:

GET  /v2/zones/tasks/transfer_accepts HTTP/1.1
Host: 127.0.0.1
Accept: application/json
Content-Type: application/json

Example Response:

HTTP/1.1 200 OK
Content-Length: 649
Content-Type: application/json; charset=UTF-8

    {
           "transfer_accepts": [
               {
                  "status": "COMPLETE",
                  "zone_id": "417fb114-df97-46e5-ba34-6f5cee925e49",
                  "links": {
                       "self": "http://127.0.0.1:9001/v2/zones/tasks/transfer_accepts/bf92e7c8-8e21-40c7-8076-8285b035b1ad",
                       "zone": "http://127.0.0.1:9001/v2/zones/417fb114-df97-46e5-ba34-6f5cee925e49"
                  },
                  "created_at": "2016-02-04T04:01:18.000000",
                  "updated_at": "2016-02-04T04:01:18.000000",
                  "key": null,
                  "project_id": "0469230b787b4154922c2a4a4b5fbcaa",
                  "id": "bf92e7c8-8e21-40c7-8076-8285b035b1ad",
                  "zone_transfer_request_id": "17bf5ba6-053f-4f1e-87c8-a2f8c3585d8f"
               }
           ]
           "links": {
               "self": "http://127.0.0.1:9001/v2/zones/tasks/transfer_accepts"
           },
           "metadata": {
                "total_count": 1
           },
    }

Central Changes

None

Storage Changes

None

Other Changes

Cli Impact:

A new cli will also be added that will enable users to list the status of all the accepted zone transfer i.e.

openstack zone transfer accept list [-h]
[-f {csv,json,table,value,yaml}] [-c COLUMN] [–max-width <integer>] [–noindent] [–quote {all,minimal,none,nonnumeric}]

List accepted zone transfers

optional arguments:

-h, --help

show this help message and exit

output formatters:
output formatter options

-f {csv,json,table,value,yaml}, –format {csv,json,table,value,yaml}
the output format, defaults to table

-c COLUMN, --column COLUMN

specify the column(s) to include, can be repeated

table formatter:

--max-width <integer>

Maximum display width, 0 to disable

json formatter:

--noindent

whether to disable indenting the JSON

CSV Formatter:

–quote {all,minimal,none,nonnumeric}
when to include quotes, defaults to nonnumeric

Alternatives

None

Implementation

Assignee(s)

Primary assignee:: Sonu kumar https://launchpad.net/~sonu-bhumca11
IRC Nick Name:: sonuk

Milestones

Target Milestone for completion:: Newton

Work Items

Add API changes
Implement CLI changes
Add the documentation for the same

Dependencies

Reference:: https://bugs.launchpad.net/python-designateclient/+bug/1499539

Expose recordsets endpoint

Fri, 18 Mar 2016 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/expose-recordsets-api

Problem description

Currently the /recordsets endpoint is a sub-resource of /zones. But there is a need to list all the records for a tenant in a single API call, so that doing filtering across all the records under a tenant will be much easier.

Proposed change

Designate will have a new API endpoint /v2/recordsets.

A single recordset can be retrieved via a GET call to /v2/recordsets/{recordset_id}, which reponds a 301 and redirects to the canonical location /v2/zones/{zone_id}/recordsets/{recordset_id}. The “self” link in the response body points to that location as well.
All recordsets across all the zones owned by a tenant can be listed via a GET call to /v2/recordsets, response will be paginated in this case if necessary.
Filtering on all recordsets under a tenant will be supported, for example /v2/recordsets?name=foo.

API Changes

API changes will be mainly about exposing the new API endpoint in controllers.

Example single recordset retrieval request:

GET /v2/recordsets/f7b10e9b-0cae-4a91-b162-562bc6096648 HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json
Content-Type: application/json

{
  "description": "This is an example recordset.",
  "links": {
      "self": "https://127.0.0.1:9001/v2/zones/2150b1bf-dee2-4221-9d85-11f7886fb15f/recordsets/f7b10e9b-0cae-4a91-b162-562bc6096648"
  },
  "updated_at": null,
  "records": [
      "10.1.0.2"
  ],
  "ttl": 3600,
  "id": "f7b10e9b-0cae-4a91-b162-562bc6096648",
  "name": "www.example.org.",
  "zone_id": "2150b1bf-dee2-4221-9d85-11f7886fb15f",
  "zone_name": "example.org.",
  "created_at": "2014-10-24T19:59:44.000000",
  "version": 1,
  "type": "A"
}

Example all recordset of a tenant retrieval request:

GET /v2/recordsets HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json
Content-Type: application/json

{
  "recordsets": [
      {
          "description": null,
          "links": {
              "self": "https://127.0.0.1:9001/v2/zones/2150b1bf-dee2-4221-9d85-11f7886fb15f/recordsets/65ee6b49-bb4c-4e52-9799-31330c94161f"
          },
          "updated_at": null,
          "records": [
              "ns1.devstack.org."
          ],
          "action": "NONE",
          "ttl": null,
          "status": "ACTIVE",
          "id": "65ee6b49-bb4c-4e52-9799-31330c94161f",
          "name": "example.org.",
          "zone_id": "2150b1bf-dee2-4221-9d85-11f7886fb15f",
          "zone_name": "example.org.",
          "created_at": "2014-10-24T19:59:11.000000",
          "version": 1,
          "type": "NS"
      },
      {
          "description": null,
          "links": {
              "self": "https://127.0.0.1:9001/v2/zones/2150b1bf-dee2-4221-9d85-11f7886fb15f/recordsets/14500cf9-bdff-48f6-b06b-5fc7491ffd9e"
          },
          "updated_at": "2014-10-24T19:59:46.000000",
          "records": [
              "ns1.devstack.org. jli.ex.com. 1458666091 3502 600 86400 3600"
          ],
          "action": "NONE",
          "ttl": null,
          "status": "ACTIVE",
          "id": "14500cf9-bdff-48f6-b06b-5fc7491ffd9e",
          "name": "example.org.",
          "zone_id": "2150b1bf-dee2-4221-9d85-11f7886fb15f",
          "zone_name": "example.org.",
          "created_at": "2014-10-24T19:59:12.000000",
          "version": 1,
          "type": "SOA"
      },
      {
          "name": "jjli.com.",
          "id": "12caacfd-f0fc-4bcb-aa24-c42769897822",
          "type": "SOA",
          "zone_name": "jjli.com.",
          "action": "NONE",
          "ttl": null,
          "status": "ACTIVE",
          "description": null,
          "links": {
              "self": "http://127.0.0.1:9001/v2/zones/b8d7eaf1-e5c7-4b15-be6e-4b2809f47ec3/recordsets/12caacfd-f0fc-4bcb-aa24-c42769897822"
          },
          "created_at": "2016-03-22T16:12:35.000000",
          "updated_at": "2016-03-22T17:01:31.000000",
          "records": [
              "ns1.devstack.org. jli.ex.com. 1458666091 3502 600 86400 3600"
          ],
          "zone_id": "b8d7eaf1-e5c7-4b15-be6e-4b2809f47ec3",
          "version": 2
      },
      {
          "name": "jjli.com.",
          "id": "f39c51d1-ec2c-48a8-b9f7-877d56b7b82a",
          "type": "NS",
          "zone_name": "jjli.com.",
          "action": "NONE",
          "ttl": null,
          "status": "ACTIVE",
          "description": null,
          "links": {
              "self": "http://127.0.0.1:9001/v2/zones/b8d7eaf1-e5c7-4b15-be6e-4b2809f47ec3/recordsets/f39c51d1-ec2c-48a8-b9f7-877d56b7b82a"
          },
          "created_at": "2016-03-22T16:12:35.000000",
          "updated_at": null,
          "records": [
              "ns1.devstack.org."
          ],
          "zone_id": "b8d7eaf1-e5c7-4b15-be6e-4b2809f47ec3",
          "version": 1
       },
  ],
  "metadata": {
    "total_count": 4
  },
  "links": {
      "self": "https://127.0.0.1:9001/v2/recordsets"
  }
}

Example recordset filtering request:

GET /v2/recordsets?data=192.168* HTTP/1.1
Host: 127.0.0.1:9001
Accept: application/json
Content-Type: application/json

{
  "metadata": {
    "total_count": 2
  },
  "links": {
    "self": "http://127.0.0.1:9001/v2/recordsets?data=192.168%2A"
  },
  "recordsets": [
    {
      "name": "ohoh.uyudbbgxdf.com.",
      "id": "a48588c5-5093-4585-b0fc-3e399d169c01",
      "type": "A",
      "zone_name": "uyudbbgxdf.com.",
      "action": "NONE",
      "ttl": null,
      "status": "ACTIVE",
      "description": null,
      "links": {
        "self": "http://127.0.0.1:9001/v2/zones/601a25f0-5c4d-4058-8d9c-e6a78f5ffbb8/recordsets/a48588c5-5093-4585-b0fc-3e399d169c01"
      },
      "created_at": "2016-04-04T20:11:08.000000",
      "updated_at": null,
      "records": [
        "192.168.0.1"
      ],
      "zone_id": "601a25f0-5c4d-4058-8d9c-e6a78f5ffbb8",
      "version": 1
    },
    {
      "name": "jli-1.uyudbbgxdf.com.",
      "id": "f2c7a0f6-8ec7-4d14-b8ec-2a55a8129160",
      "type": "A",
      "zone_name": "uyudbbgxdf.com.",
      "action": "NONE",
      "ttl": null,
      "status": "ACTIVE",
      "description": null,
      "links": {
        "self": "http://127.0.0.1:9001/v2/zones/601a25f0-5c4d-4058-8d9c-e6a78f5ffbb8/recordsets/f2c7a0f6-8ec7-4d14-b8ec-2a55a8129160"
      },
      "created_at": "2016-04-04T22:21:03.000000",
      "updated_at": null,
      "records": [
        "192.168.6.6"
      ],
      "zone_id": "601a25f0-5c4d-4058-8d9c-e6a78f5ffbb8",
      "version": 1
    }
  ]
}

Central Changes

Central changes will include changing functions for finding recordsets from storage in central.service to support corresponding calls from api layer.

Storage Changes

Corresponding changes to support the API change.

Other Changes

None

Alternatives

None

Implementation

Assignee(s)

None

Milestones

newton-1

Work Items

Make code changes to api, central and storage
Add unit and functional tests.

Dependencies

None

Deleted domains purging

Tue, 17 Nov 2015 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/deleted-domains-purging

Implementation of a mechanism to routinely purge deleted domains from the database.

Problem description

Once deleted, domains are not removed immediately from the database, mostly for billing reasons. They are flagged as deleted in the “deleted” database column and the “deleted_at” column is populated with a timestamp.

Currently, deleted domains are purged manually by running SQL queries against the database. The purging process impacts the database performance and requires manual intervention.

Proposed change

This change is to implement an automated purging mechanism for deleted domains.

It also provides “purge_domain” and “purge_domains” calls to purge domains using an arbitrary criterion.

API Changes

Central API bumps to version 5.3 with the addition of the “purge_domains” call.

Zone Manager Changes

Implement purging as a periodic task in Zone Manager using a “domain_purge” plugin. The task will select a group of domains and send a RPC call to Central. Central will run a query against the database to purge any deleted domain if needed and log the number of purged domains.

Configuration parameters:

Purging run frequency.: Default: hourly. Users might want to run it frequently to minimize the cycle duration.
Per-cycle purging limit.: Default: 100.
Time threshold.: Default: 7 days.

Configuration example:

# Deleted domains purging
#------------------------
[zone_manager:domain_purge]
# How frequently to purge deleted domains, in seconds
#run_interval = 3600  # 1h
# How many deleted domains to be purged on each run
#limit = 100
# How old deleted domains should be to be purged, in seconds (deleted_at column)
#time_threshold = 604800  # 7 days

Central Changes

Implement purge_domains() method.

Storage Changes

Implement purge_domain(), purge_domains() method.

Update the _delete() method to allow forced hard delete.

Other Changes

Add a new task to setup.cfg and a new purge_domains policy item to policy.json

Add some test utility functions.

Alternatives

No change: keeping the current manual process in place.
Deleting+purging the domains immediately.
Purging the domains on a record creation/deletion event.
Purging deleted domains with a single SQL query.

Further improvements

Domains for purging are queried with a column comparison “deleted_at > $date”

Query performance can be improved by indexing the “deleted_at” column as a B-Tree (supported by InnoDB and MyISAM): https://dev.mysql.com/doc/refman/5.0/en/index-btree-hash.html

Implementation

Assignee(s)

Primary assignee:: https://launchpad.net/~federico-ceratto

Milestones

Target Milestone for completion:: Liberty-3

Work Items

Implement purging
Update documentation
Implement benchmarks

Dependencies

TODO

Bulk zone update throttling

Tue, 17 Nov 2015 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/notify-throttling

Implement a mechanism to throttle the delivery of NOTIFY transactions when a large number of zones are updated at the same time.

Problem description

If a large number of zones are updated in a short time this will generate a consequently large amount NOTIFY transaction to be sent to the nameservers with no delay leading to a burst of incoming AXFR requests. This might impact on bottlenecks in MiniDNS and the storage layer in terms of CPU, I/O or network bandwidth.

A typical trigger is the update of an NS record in a Pool containing many zones.

The autonomous refreshing of zones performed by resolvers can also trigger a similar burst of AXFR. This can happen on recently started resolvers, where the refresh timers can share the same values across many zones.

Proposed change

Implement a mechanism for enqueuing and delayed delivery of notify transactions at a configurable throttle speed.

Also, implement staggering of zone refresh requests by randomizing the refresh interval.

API Changes

Expose the count of zones flagged for delayed notify in the Admin API as “/reports/counts/zones_pending_notify”.

Central Changes

Implement support for a new database column “pending_notify” and set it to True every time a Pool NS record is updated.

Storage Changes

Add an new boolean database column “pending_notify” on Zones. Implement a migration script to add the column to existing databases, defaulting to False. In future, the column might default to True.

Other Changes

Implement a Task in Zone Manager to periodically fetch a set of zones that need to receive a Notify starting with the oldest in term of last update time. The task frequency and the maximum set size can be configured to throttle the amount of outgoing Notify. Zone Manager will reset the “pending_notify” flag once done.

Alternatives

N/A

Implementation

The throttling queue is implemented as a new database column containing a boolean flag. See Central Changes and Storage Changes.

Also, new zones will be created with an uniformly random refresh time between a minimum and a maximum value.

Design considerations

The throttling queue could be implemented outside of the database: - No need to create an extra database column - No increased database I/O

We propose using the database for the following reasons: - Zone Manager is the best candidate to handle the delayed Notify. Currently there are no ways for Central to send a list of Zones to Zone Manager other than through the database - The queue can support delayed Notify for changes other than Pool NS record updates - Ability to monitor the queue size and ETA to inform the user and for debugging - A persistent queue can survive Zone Manager unhandled exceptions or restarts - The increased database load is negligible compared to the existing traffic

Risk analysis

Zone Manager fails to run the Notify delivery task. The nameservers will eventually refresh the zone anyways. Impact: slow update propagation. Mitigation: expose the notification queue length to the user through Admin API and by logging.
A big notification queue takes a considerable time to be handled. Impact: potentially prevents more urgent changes to be delivered quickly. Mitigation: encourage users to configure the throttling parameters; Provide sensible default values. Implementing a concept of notification priority seems unnecessary.

Assignee(s)

Primary assignee:: Federico Ceratto https://launchpad.net/~federico-ceratto

Milestones

Target Milestone for completion:: Liberty-3

Work Items

Implement refresh time staggering
Implement Notify throttling
Add throttle parameters to configuration files
Document throttling mechanism
Write unit and functional tests
Test throttling and staggering on devstack

Dependencies

N/A

Designate to Designate driver

Mon, 17 Aug 2015 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/d2d-driver

Add a driver that uses a Designate service and SECONDARY zones like we do with Dyn / Akamai.

Problem description

We want to be able to use a instance of a Designate service to act as the target for our zones.

Proposed change

Create a driver that utilizes the SECONDARY zones feature of Designate.

API Changes

N/A

Central Changes

N/A

Storage Changes

N/A

Other Changes

Add driver ‘designate’ to designate.backends using v2 api and secondary zones. Authentication is done via the keystoneclient auth / session using the provided credentials in the backend options.

Alternatives

N/A

Implementation

Assignee(s)

Primary assignee:: endre-karlson <endre.karlson at hp dot com>

Milestones

Target Milestone for completion:: Liberty-3

Work Items

Create designate.backend.impl_designate with associated unit tests and devstack support

Dependencies

N/A

Hook Points API

Wed, 08 Apr 2015 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

Hook Point API Blueprint

Designate, by design, is required to interface with legacy DNS applications. Organizations will typically have other systems that must interface with DNS and vice versa. The hook points API provides a generic way to add organization specific code that adds no overhead in the default use case and does not allow changes to the underlying APIs when code is hooked in to a method or function call.

Problem description

Currently, in order to support organization specific functionality, it is required to either:

Maintain a fork of the Designate code base

Monkey patch the code via a module

Treat designate as a black box and use HTTP level tools to add org specific code.

While maintaining a fork can be a reasonable solution, if there are large changes, it can be very difficult to merge upstream changes. Monkey patching forces the same level of knowledge regarding the code base, yet it is tricky to get right as small code changes can silently fail due to import changes. Lastly, treating Designate as a black box will often require org specific code to reimplement aspects of Designate due to limited level of granularity.

The Hook Points API provides a practical supported means of injecting code, while avoiding the pitfalls mentioned above.

Proposed change

The Hook Points API provides a decorator to define a function or method as a hook point.

@hookpoints.hook_point('pool_manager_create_domain')
def create_domain(self, context, domain):
    ...

In the above case the hook point is a named hook point called pool_manager_create_domain. When a name is not provided, the hook point uses the path of the module and method.

@hookpoints.hook_point()
def create_domain(self, context, domain):
     ...

The name is the combination of function’s module and name.

'%s.%s' % (func.__module__, func.__name__)

If no hook point is defined, the original function is returned as-is.

In order to define a hook point, a package must be installed that provides a designate.hook_point entry point.

from setuptools import setup


setup(
    name='raxdns',
    entry_points: {
        'designate.hook_point': [
            'pool_manager_create_domain = raxdns.hooks.pool_manager:create_domain'
        ]
    }
)

When the package is installed, the hook point is then active meaning that it will be called when the target is called. It may be disabled via the configuration if necessary.

Hook Point Implementation

A hook point, as it will be applied as a decorator, is an object that implements a __call__ method that accepts a single function as the argument and returns an appropriate function as the result. It is up to the implementor to ensure the hook point correctly implements the API of the hook target.

For convenience, there is BaseHook that can be used to reuse common patterns.

class BaseHook(object):

    OPTS = [
        cfg.BoolOpt('disabled', default=False)
    ]

    def __init__(self, group):
        self.group = group

    @property
    def disabled(self):
        return cfg.CONF[self.group].get('disabled', False)

    def wrapper(self, *args, **kw):
        return self.hook_target(*args, **kw)

    def __call__(self, f):
        # Save our hook target as an attribute for our wrapper method
        self.hook_target = f

        @functools.wraps(self.hook_target)
        def wrapper(*args, **kw):
            if self.disabled:
                return self.hook_target(*args, **kw)
            return self.hook(*args, **kw)
        return wrapper

The BaseHook takes care of:

using functools.wrap correctly

disabling the hook when configured to do so

setting the config group for later use of the config

simplifying the decorator implementation

This base class is meant to make development of a hook simpler. A hook author is free to implement the hook as a normal decorator as well.

Configuration

It is important to note that any configuration must be accessed via oslo.config. The reason being is that the hooks are applied at import time, where the confguration is typically loaded at run time. Therefore, the hook may not have access to config data until the hook target is actually called.

Hook Example

Here is an example of a hook point that wraps the create_domain method in the Pool Manager service. It validates the domain doesn’t exist in another application that can also manage domains via the same backends.

import requests
from oslo_log import log as logging
from oslo_config import cfg

from designate.pool_manager.service import ERROR_STATUS
from designate.hookpoints import BaseHook


LOG = logging.getLogger(__name__)


class CheckDCXDomainHook(BaseHook):
    OPTS = BaseHook.OPTS + [
        cfg.Opt('legacy_dns', required=True),
    ]

    @property
    def sess(self):
        if not hasattr(self, '_sess'):
            sess = requests.Session()
            self._sess = sess
        return self._sess

    @property
    def legacy_dns(self):
        return cfg.CONF[self.group].legacy_dns

    def hook(self, obj, context, domain):
        resp = self.sess.get(self.legacy_dns + '/find_domains?name=%s' % domain.name)

        # The domain is not found in the legacy system. Let Designate create it
        if not resp.ok:
            # We got a 404, so let Designate make the call
            return self.hook_target(obj, context, domain)

        # The legacy system owns the domain. Notify central it was
        # an error.
        #
        # The `obj` is the Service object.
        obj.central_api.update_status(
            context, domain.id, ERROR_STATUS, domain.serial
        )

It is the responsibility of the hook point author to be a good citizen and properly handle any errors / return values in the original code, as well as support any internal APIs.

Again, the intent of the Hook Point API is to allow an organization a means of injecting code, which implies a reasonably intimiate knowledge with the code.

Hook Point Management and Configuration

Hook points are installed by installing a package that includes designate.hook_point entry points. By default, these will be enabled and will be called when the specific hook point target is called. These hooks MAY be disabled in the config at the hook point level.

[hook_point:pool_manager_create_domain]
disabled = True

If necessary, hook points can also receive configuration details.

[hook_point:pool_manager_create_domain]
legacy_dns_api = https://my.dns.legacy.org.net:8975

The configuration will be available via the global oslo.config.cfg.CONF object.

Central Changes

None

Storage Changes

None

Other Changes

Hook points can be added liberally or in an extremely limited, known uses cases. Similarly, no hook points can be formally added and an organization may apply them as necessary in an org specific patch.

Alternatives

Other than the originally mentioned tactics for hooking into the Designate code, more specific hook points could be created on a per-use basis. For example, there could be very a specific API for hooks that get called when a message is read from the queue. While providing more specific hooks may allow for a use case specific API, it would also require each API be designed, documented and tested. The hook point API provides a single tested means of injecting code that has limited effect on the API over time and allows a reasonable level of support.

Implementation

Assignee(s)

Primary assignee:: eric-larson

Milestones

Target Milestone for completion:: Liberty-1

Work Items

Add designate.hookpoints, implementing the @hook_point decorator

Add documentation for writing hook points and enumrate what hook points exist.

See this review for the current implementation.

dependencies

stevedore

Graded Backends

Wed, 01 Apr 2015 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/graded-backends

Problem description

Currently we have quite a few backends, that are not tested, or actively maintained. We should be informing users of what backends are most actively maintained and tested

Proposed change

A graduated level of grading for backends:

Grade	Variations	Description
Integrated	None	Tested on every commit by the OpenStack CI Infrastructure, and maintained by designate developers as a reference backend
Master Compatible	In Tree / External	Tested on every commit by 3rd party testing, and has a person or group dedicated to maintaining compatibility on a regular basis
Release Compatible	In Tree / External	Not necessarily tested on every commit, but has a maintainer committed to ensuring compatibility for each release
Untested	In Tree / External	All other backends in the designate repository
Failing	In Tree / External	Backends that were previously “* Compatible”, but tests are now failing on a regular basis.
Known Broken	In Tree / External	Backends that do not work, and have been broken with no sign of any fixes

The current backends would be currently fall into the following pattern:

Backend	Grade	Notes
PowerDNS + MySQL	Integrated
Bind	Integrated
Akamai	Release Compatible - In Tree
DynECT	Release Compatible - In Tree
Agent	Untested - In Tree
PowerDNS + pgSQL	Untested - In Tree
Microsoft DNS	Untested - External

This will also include the creation of a support matrix like Nova

This info should be maintained along with the list of current driver maintainers responsible for the “Non Integrated” backends. The upkeep of this list will fall on the PTL or his/her delegate

Should a backend’s grade be in dispute, it falls on the current project PTL to make the final decision after listening to all sides concerns.

Alternatives

Continue on as is

Implementation

Assignee(s)

Primary assignee:: grahamhayes

Milestones

Target Milestone for completion:: Liberty-1

Work Items

Dependencies

None

Introduce Guru Meditation Reports into Designate

Mon, 09 Mar 2015 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/guru-meditation-reports

Problem description

Guru Meditation Reports enables Designate service to print a lot of detail debug information about service running thread, configuration and greenlet stack straces when receiving USR1 signal.

Add Guru Meditation Reports into Designate will help us debugging problems easier especially when dealing with bugs root in deadlocks between eventlet greenthreads.

Guru Meditation Reports is alreadly implemented in oslo-incubator and used in Nova.

Proposed change

Pick Guru Meditation Reports implementation from oslo-incubator and add the USR1 signal handler.

All Designate services will support Guru Meditation Reports, including:

designate-central
designate-api
designate-mdns
designate-agent
designate-pool-manager
designate-sink
designate-manage

API Changes

None

Central Changes

None

Storage Changes

None

Other Changes

Designate service processes will receive USR1 signal and print Guru Meditation Reports.

Implementation

Assignee(s)

Primary assignee:: stanzgy

Milestones

Target Milestone for completion:: Kilo-3

Work Items

Work items are listed in Proposed change section.

Documentation Impact

Describe how to generate and view a Guru Meditation Reports.

References

GMR wiki: https://wiki.openstack.org/wiki/GuruMeditationReport
GMR in Nova: http://docs.openstack.org/developer/nova/devref/gmr.html
Nova GMR User Manual: http://docs.openstack.org/admin-guide-cloud/content/section_compute-GuruMed-reports.html

Zone Import Refactor

Wed, 29 Oct 2014 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

Migrate Zone import code from designate.api.v2.controllers.zones into a new designate.dnsutils module and move functionality into Central.

Terminology

Term	Meaning
zone	DNS Zone (also refered to as a DNS domain)

Problem description

Currently the Import functionality parses the Zone that is POST’ed in the API and runs the creation of the Zone with its Records from the API side. There is no transactional guarantee of a cleanup if the creation of anything fails.

Also there’s a given value of having these methods outside of the API, it can be used in any component outside of the API (for example AXFR support).

Proposed change

We introduce a new module called designate.dnsutils that contains functions for parsing Zones / Records into dnspython object and dnspython objects to DesignateObjects.

Code for parsing a Zone file is already present in the designate.api.v2.controller.ZoneController and can be easily extracted.

Central Changes

We need a new method that takes a Domain and a RecordSetList object, then puts this into Designate accordingly.

It should have the ability to either synchronize a existing zone or at a later point in time a new zone with it’s records.

New module dnsutils

dnsutils.from_dnspython_zone(dnspython_zone)

Takes a dnspython_zone and creates a Domain object also runs records_to_recordset_list() on each record in the zone.

Parameter	Description	Required
dnspython_zone	DNS Python zone	Yes

Return value

Domain with a recordset relation attribute

dnsutils.dnspyrecords_to_recordsetlist(dnspython_records)

Convert a dnspython_zone.nodes structure to a RecordSetList, Recordsets, RecordList and Records.

Parameter	Description	Required
dnspython_records	DNS Python Record	Yes

Return value

designate.objects.recordset.RecordSet with a record attribute of designate.objects.record.RecordList from the given dnspython node / record.

dnsutils.dnspythonrecord_to_recordset(rname, rdataset)

Parameter	Description	Required
rname	dnspy node name / key	Yes
rdataset	dnspy rr dataset	Yes

Return value

designate.objects.recordset.RecordSet with a record attribute of designate.objects.record.RecordList from the given dnspython node / record.

Implementation

Assignee(s)

Primary assignee:: <https://launchpad.net/~endre-karlson>

Milestones

Target Milestone for completion:: Kilo-1

Work Items

Extract code from Zones controller into new module “dnsutils” / new central method.
Change parse_zonefile to call dnsutils.parse_zonefile() and call Central’s new method with the results of this.

New Agent

Tue, 28 Oct 2014 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

This spec describes a proposed service for Designate, tentatively called the “Agent,” not to be confused with the current component with the same name.

Terminology

Term	Meaning
AXFR	A DNS zone transfer
NOTIFY	A DNS protocol to inform of a zone update
OPCODE	Part of a DNS message that informs action http://bit.ly/1pfG8Zp
Backend	Different code paths for various DNS servers
MiniDNS	Designate component in charge of sending NOTIFYs, and answering AXFRs.

Problem description

MiniDNS provides a very elegant solution for many DNS installations. Having a Designate component as the DNS master enables Designate to have a great amount of control over the DNS server, and makes certain processes much easier to implement across backends.

However, some deployments may not be able to use MiniDNS as a true DNS Master, and would rather keep the current style of backends in some form with the added benefits of MiniDNS. They might need a more specialized type of interaction between Designate and the DNS server, making Designate more flexible. Additionally, there are potential problems with having MiniDNS and it’s database being a single point of failure for a DNS infrastructure.

Proposed change

An agent deployed on the Master DNS server that interacts with MiniDNS solves these problems. A plugable backend that enables users to use any backend with MiniDNS, as well as the ability to isolate MiniDNS from being a master in large deployments are much simpler with an Agent.

The Agent would be a standalone service that acts as a sort of mirror to MiniDNS. It would receive AXFR/IXFR, NOTIFYs, and other notifications and perform changes to the DNS server through a plugin-style backend.

Pool Manager Changes

A new backend for the Pool Manager would be created for the Agent. It should send fire-and-forget DNS messages with classes and RRTYPE intended for private use, shown here: http://bit.ly/1soCZTffor on Creates/Deletes, and call into MiniDNS as it would normally for updates.

Agent Changes

The agent service will need to be created (possibly supplanting the old “agent”)

The structure will be very similar to MiniDNS. A service (TCP, rather than RPC) will listen for TCP and UDP traffic. NOTIFYs and AXFRs will be handled, along with regular DNS queries (via pass through to the real DNS server), and the special DNS CLASSES and RRDTYPEs that are chosen to signify Creates and Deletes. DNS traffic will result in calls into the configured backend. A special OPCODE, 14 will be used for all non-standard actions like Creating and Deleting zones.

Configuration

Parameter	Default	Required	Notes
backend_driver	‘None’	Yes	The name of the backend driver to use with the Agent
workers	None	No	The number of worker processes to spawn
masters	[]	Yes	List of IP/Ports of Masters the Agent will query for AXFRs and SOA queries
allow-notify	[]	Yes	List of IP/Ports of Masters allowed to NOTIFY/AXFR with the Agent
host	‘0.0.0.0’	Yes	Bind host for the Agent
port	5354	Yes	Port to bind to
tcp_backlog	100	Yes	TCP backlog for the Agent

There would also be sub-configuration sections for the different backends, using the form [agent:backend_name]. These would be mostly ported from the current backends.

Service

Initialization of the service, and basic handling of TCP/UDP traffic. Does some validation on the traffic coming in, making sure it is valid before sending on to the core application logic. This should be mostly taken from designate/mdns/service.py There could be some optimization, since the Agent will be talking only to mdns.

Middleware

A thin middleware that sits between the TCP message receiving, and the handler. Allows for contexts or other metadata to be attached to a request before it goes to a handler.

Handler

The handler will take DNS requests in, and take the appropriate action. This will be very similar to mdns/handler.py

_handle_update(request, op)

Parameter	Description	Required
request	Serialized request data from the DNS request	Yes
op	Whether an update or create is happening	Yes

Get the name from the request
Make sure the requester is in the allowed list of notifiers
Call the backend’s find_domain to see if the message is for a zone on the DNS server. If not, throw the message away
Query the requesting mdns (SOA), per the RFC (????)
Call out to the Agent’s AXFR module asynchronously which will perform the AXFR, get the zone and call backend’s create or update domain, based on the action.

_handle_delete(request)

Parameter	Description	Required
request	Serialized request data from the DNS request	Yes

Get the name from the request
Make sure the requester is in the allowed list of notifiers
Call out to the backend’s delete_domain

_handle_record_query(request)

Parameter	Description	Required
request	Serialized request data from the DNS request	Yes

Repackage the request, send it along to the local DNS server
Return the results

This is a possible way for the Agent to answer DNS queries when MiniDNS is polling for changes

AXFR

The AXFR module will send the AXFR query to one of the masters specified in the config for the zone name that was passed in. Based on the type of query that called for the AXFR, a backend call will then be made with the information gathered from the AXFR.

_do_axfr(zone_name, action)

Parameter	Description	Required
zone_name	The zone name to ask for the AXFR with	Yes
new_domain	Boolean value to inform AXFR if the zone is a new zone or not. Default is False	No

Pick a master from the config file
Send the AXFR query
Parse the response into a designate domain object
Call the backend for the specified action (Create, Update)

Note

Eventually this module could be renamed “Transfer” and do IXFRs.

Backend

The Backend module will house a base plugin, and a variety of plugins with a similar interface to the Pool Manager. The intent is to take the zone changes gleaned from an AXFR (or IXFR) and apply them to the DNS server. The manner of accomplishing this will vary widely for each DNS server, you might be editing a flat file, or making database calls, or some other method. The following methods would compose the base plugin.

find_zone(zone_name)

Parameter	Description	Required
zone_name	The zone name to be searched for	Yes

When a NOTIFY comes in for a zone, the Agent must first check to make sure that the zone is valid for the DNS server. Otherwise, it might do an AXFR and try to update a zone that doesn’t exist on the server. It’s possible that this check could be incorporated in to the update_zone function, but it seems more efficient to do it here.

create_zone(domain)

Parameter	Description	Required
domain	A Designate domain object to create on the backend server, including it’s recordsets	Yes

Take the appropriate measure to create the zone on the DNS server. This object will hold all the necessary information to serve the zone.

update_zone(domain)

Parameter	Description	Required
domain	A Designate domain object to update on the backend server, including it’s recordsets	Yes

Take the appropriate measure to update the zone on the DNS server. This has the potential to be very similar to the create_zone logic if there is no way to discern the differences between this object, and the zone on the server.

delete_zone(zone_name)

Parameter	Description	Required
zone_name	The zone name identified for deletion	Yes

Take the appropriate measure to delete the zone on the DNS server, and ideally all of it’s subresources.

Other Changes

This should fit into the Designate pattern well. To the Pool Manager and MiniDNS, the Agent is the same as any other DNS server. It’s possible that something in MiniDNS could be supplemented to use with the Agent, but it shouldn’t be needed.

A lot of the code that MiniDNS uses for the actual DNS protocol stuff will be reused in some form in the Agent. Development of the agent would be a good time to port the commonalities into the dnsutils module of Designate.

Benefits

Configurable backends that can do the work required for different DNS servers (RNDC, Database addition) that don’t connect directly to the database
A deployment with the benefits of MiniDNS, while keeping a traditional Master/Slave DNS setup is possible
Less DNS servers must be managed by Designate
Less chatter between MiniDNS and the database because SOA refreshes and other queries can be handled by a master
MiniDNS becomes less vital to a deployment, MiniDNS/Database issues are isolated from the DNS infrastructure
An agent with direct control of the DNS servers adds the benefit of doing things outside of pure DNS protocol with a DNS server, periodic syncs, etc are made easier

Implementation

Assignee(s)

Primary assignee:: tim-simmons-t

Milestones

Target Milestone for completion:: Kilo

Work Items

Create the Agent service
Add support for receiving NOTIFYs
Add support for receiving AXFRs
Decide and implement receiving messages with non-standard CLASS/RRDATA
Add a base class backend that is called for different operations
Port some of the existing backends, or add some new ones

Support for server pools in Minidns

Wed, 01 Oct 2014 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/server-pools-minidns-support

This spec outlines the apis that need to be added to MiniDNS for supporting pool manager.

Problem description

notify_zone_changed: The pool manager needs to notify pool servers when a zone changes. It calls into mindns to notify the pool servers of the zone change.
poll_for_serial_number: When a zone is changed, the change is marked as PENDING in central. It will be marked as ACTIVE when a certain number of pool servers have the change. Server pools would use mindns to check the pool servers for the presence of the change.

Proposed change

There would be 2 calls added to minidns to support the features needed by pool manager.

Mindns Call Changes

notify_zone_changed(context, domain, destination, timeout, retry_interval, max_retries)

Parameter	Description	Required
context	The user context.	Yes
domain	The designate domain object.	Yes
destination	The recipient of the NOTIFY message. The format is “ip:[port]”. If there is no port, port 53 is used.	Yes
timeout	The time to wait for a NOTIFY response from destination.	Yes
retry_interval	The time between retries.	Yes
max_retries	The maximum number of retries mindns would do for a NOTIFY response. After this many retries, mindns drops the NOTIFY message.	Yes

Design Considerations

Mindns uses domain.name to construct a NOTIFY message and sends it to destination. If a response is not received within timeout interval, then after a time interval of retry_interval, the NOTIFY message is resent to destination. After trying for max_retries, the call completes. All times are in seconds.

Cast vs Call

This is an RPC cast rather than an RPC call. The pool manager need not wait for the NOTIFY response. The reasoning here is that in the worst case after the refresh interval, the zone on the destination would be refreshed. Also the status would be correctly updated elsewhere so this need not be a call.

Single Call vs Multiple Calls

Pool Manager sends multiple requests to (potentially multiple instances of) minidns with a single destination in each call. The reasoning is that the pool manager has the global knowledge of the number of pool servers and NOTIFY messages that need to be sent. It is in a better position to decide the approach to parallelizing the messages that need to be sent. The minidns on the other hand is only concerned with the dns protocol. That said BIND sends a zone transfer request to the server that sends the NOTIFY message. So it makes sense to have a minidns server associated with the same set of pool servers.

Parameter sources/validations

The timeout, retry_interval and max_retries would possibly be specified in the config file in the pool manager section for the initial version. Irrespective of where the pool manager gets its data, it passes these as parameters to minidns. It makes sense for pool manager to track these parameters rather than minidns.

Minidns does not do any validation of the timeout, retry_interval and max_retries parameters.

poll_for_serial_number(context, domain, destination, timeout, retry_interval, max_retries)

Parameter	Description	Required
context	The user context.	Yes
domain	The designate domain object.	Yes
destination	The server to check for an updated serial number. The format is “ip:[port]”. If there is no port, port 53 is used.	Yes
timeout	The time to wait for a SOA response from destination.	Yes
retry_interval	The time between retries.	Yes
max_retries	The maximum number of retries mindns would do for an expected serial number. After this many retries, mindns returns an ERROR.	Yes

Design Considerations

Mindns uses domain.name to construct a SOA request and sends it to destination. The serial number in the SOA response (hereafter referred to as actual_serial) is compared to domain.serial (hereafter referred to as expected_serial).

Cast vs Call

This would be a cast. Pool Manager would provide a function - update_status for minidns to inform of the result. For more discussions on this see the designate meeting log notes on 03-Sep-2014

Return Values

The return status from this function is sent to the pool manager using the pool manager call back function - update_status. The values sent are summarized below. We actually could send just the serial number without a status too.

Case	Action
actual_serial = expected_serial	Send SUCCESS and actual_serial.
actual_serial > expected_serial	Send SUCCESS and actual_serial.
actual_serial < expected_serial	poll again after a time interval of retry_interval elapses. Continue this until we get the expected_serial or for max_retries. Send ERROR and actual_serial after max_retries
timeout for max_retries	Send ERROR and None for serial number.

Pool Manager Dependencies

Pool Manager would provide an api - update_status for minidns to call in to inform the status of the poll_for_serial_number call. The Return Values section of poll_for_serial_number summarizes the values that would be sent with update_status As noted above, we need not send a separate status, sending the actual_serial (or None) should be enough.

update_status(context, domain, destination, status, actual_serial)

Parameter	Description	Required
context	The user context passed in the corresponding . poll_for_serial_number call	Yes
domain	The designate domain object. domain.serial has the expected_serial number. This object is the same as the one sent in the corresponding poll_for_serial_number call	Yes
destination	The destination parameter from the corresponding poll_for_serial_number call	Yes
status	The status would be one of SUCCESS, ERROR.	Yes
actual_serial	The actual serial number received from destination or None in case of a timeout.	Yes

Storage Changes

None

Implementation

Assignee(s)

Primary assignee:: <vinod-mang>

Milestones

Target Milestone for completion:: Kilo

Work Items

Api addition to mini-dns - notify_zone_changed
Api addition to mini-dns - poll_for_serial_number
Api to pool manager (minidns would call this api) - update_status

Dependencies

Pool Manager needs to add update_status for minidns to inform the status of poll_for_serial_number.

Server Pools Storage

Mon, 11 Aug 2014 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/server-pools-storage

This specification outlines the Central, Storage, and compatibility changes needed to support server pools.

Problem description

For server pools, a new object is required to encapsulate pools. When this specification is implemented, the pool object will used by the backend drivers for getting the name servers as a compatibility change. Central will continue to call the backend drivers. Additional server pools specifications will address using the entire pool object. This is the foundation for those other specifications.

Proposed change

When the database is first initialized, a pool named ‘default’ will be created. This is the default pool. The default pool will have no also-notify hosts and no name servers. New methods will be added to Central to create, find, get, update, and delete pools and these methods are described in the Central Changes section. The database changes needed to support pools are described in the Storage Changes section. Finally, name servers can be added to the default pool by using the V1 servers endpoint described in the Compatibility Changes section.

API Changes

None

Central Changes

The [service:central] section of the Designate configuration will have a new option:

Parameter	Default	Required	Notes
default_pool_name	‘default’	Yes	The pool name of the default pool

This value needs to be set to the pool name of the default pool which is ‘default’.

The pool object will be implemented.

The new methods implemented in the Central service for the pool object are:

create_pool(context, pool)

Parameter	Description	Required
context	Security context information.	Yes
pool	The designate pool object.	Yes

Return Value

The created pool.

find_pools(context, criterion, marker, limit, sort_key, sort_dir)

Parameter	Description	Required
context	Security context information.	Yes
criterion	The search criterion.	No
marker	Resource ID from which after the requested page will start after.	No
limit	Integer limit of objects of the page size after the marker.	No
sort_key	Key from which to sort after.	No
sort_dir	Direction to sort after using sort_key.	No

Return Value

The found pools.

get_pool(context, pool_id)

Parameter	Description	Required
context	Security context information.	Yes
pool_id	The pool ID.	Yes

Return Value

The pool requested.

update_pool(context, pool)

Parameter	Description	Required
context	Security context information.	Yes
pool	The designate pool object.	Yes

Return Value

The updated pool.

delete_pool(context, pool_id)

Parameter	Description	Required
context	Security context information.	Yes
pool_id	The pool ID.	Yes

Return Value

No return value.

Storage Changes

A new table for pools will be needed. Another new table for the pool attributes will be used to store the also-notify and name server values in addition to supporting future enhancements to server pools. The domains table will be modified to support pools.

New Table - pools

Column	Type	Nullable?	Unique?	Notes
id	CHAR(32)	No	Yes	PK
created_at	DATETIME	Yes	No	UTC time of creation
updated_at	DATETIME	Yes	No	UTC time of last update
version	INT(11)	No	No	Designate API version
name	VARCHAR(50)	No	Yes	Server pool name
description	VARCHAR(255)	Yes	No	Server pool description
project_id	VARCHAR(36)	No	No	Tenant ID that created the pool
provisioner	ENUM	No	No	Only ‘unmanaged’ provisioner is supported currently

New Table - pool_attributes

Column	Type	Nullable?	Unique?	Notes
id	CHAR(32)	No	Yes	PK
key	VARCHAR(255)	No	No	Pool attribute key (also-notify, name server, etc.)
value	VARCHAR(255)	No	No	Pool attribute value
pool_id	CHAR(32)	No	No	FK to ID on pools table

Modify Table - domains

Column	Type	Nullable?	Unique?	Notes	Action
pool_id	CHAR(32)	No	Yes	FK to ID on pools table	add

Compatibility Changes

When adding a name server using the V1 servers endpoint, a pool_attributes table entry will be created for the name server. The columns and values used will be:

Column	Value
key	‘name_server’
value	<FQDN of name server>
pool_id	<default pool id>

When creating a domain using the V1 domains endpoint, the domains table entry will include the pool ID of the default pool.

The backend drivers will be modified to use the name servers defined for the default pool in the pool_attributes table instead of the servers table.

All Central methods for servers will be removed.

The servers table will be removed and all Storage methods for servers will be removed.

The existing server object will be removed.

Other Changes

None

Alternatives

None

Implementation

Assignee(s)

Primary assignee:: https://launchpad.net/~rjrjr
Additional assignee:: https://launchpad.net/~darshan104

Milestones

Target Milestone for completion:: Kilo-1

Work Items

Central changes
Storage changes
Compatibility changes

Dependencies

None

Zone and Record TotalCount

Thu, 10 Jul 2014 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/zone-and-record-totalcount

Problem description

When a user makes a request for a list of zones or records, provide the total count of zones or records that match the given query. The count will ideally be provided when a user makes a request for something like /zones or /records (to limit the need to make multiple queries).

Proposed change

API Changes

This change would not add any new endpoints, but it would modify the data presented by some existing endpoints.

The following examples demonstrate the total_entries value for a generic collection of “resources”. After implementation, this feature will be supported by collections of zones and recordsets.

GET /v2/resources

Example Request:

GET /v2/resources HTTP/1.1
Accept: application/json
Content-Type: application/json

Example Response:

HTTP/1.1 200 OK
Content-Type: application/json

{
  "resources": [{
    "id": "fdd7b0dc-52a3-491e-829f-41d18e1d3ada",
    "created_at": "2014-06-23T18:39:32.000000",
    "....": "...."
  }, {
    "id": "a86dba58-0043-4cc6-a1bb-69d5e86f3ca3",
    "created_at": "2014-07-08T20:28:19.000000",
    "....": "...."
  }, {
    "id": "460f7531-e381-4773-aff3-06a12fad096d",
    "created_at": "2014-06-04t19:09:17.000000",
    "....": "...."
  }, {
    "id": "40ced622-fc70-498d-9f28-3d3021b19685",
    "created_at": "2014-07-08T16:47:32.000000",
    "....": "...."
  }],
  "links": {
    "self": "https://dns.provider.com/v2/resources?sort_key=id&sort_dir=desc"
  },
  "meta": {
    "total_entries": 4
  }
}

Central Changes

A PagedListObjectMixin class will be added that will support the metadata associated with list pagination. This class will be added as a superclass of the ZonesList and RecordsList classes.

Storage Changes

Any calls to find_domains will also internally call count_domains and add this count to the DomainsList object that is returned.

Any calls to find_records will also internally call count_records and add this count to the RecordsList object that is returned.

Other Changes

None

Alternatives

None

Implementation

Assignee(s)

Primary assignee: jordan-cazamias

Milestones

Target Milestone for completion:: Juno-2

Work Items

Agree on spec for API formatting
Implement changes

Dependencies

https://review.openstack.org/#/c/105021/

V2 API - Bindings

Sun, 06 Jul 2014 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

BP: https://blueprints.launchpad.net/python-designateclient/+spec/v2-api-bindings

Provide bindings for the V2 API using the Session based approach used for V1.

Problem description

Provide bindings for the V2 API. (This does not include the shell commands)

Proposed change

We will provide a designateclient.v2.client.Client object which would load the other Managers (not much unalike the Controllers or so in V1).

Manager - Zone

List

List Zones.

client.zones.list()

Get

Get a Zone.

client.zones.get(zone)

Create

Create a Zone.

client.zones.create(name)

Update

Update a Zone.

client.zones.update(zone)

Delete

Delete a Zone.

client.zones.delete(zone)

Manager - RecordSets

List

List Recordsets in a Zone.

client.recordsets.list(zone)

Get

Get a RecordSet

client.recordsets.get(zone, recordset)

Create

Create a RecordSet in a Zone.

client.recordsets.create(zone, name, data)

Update

Update a RecordSet.

client.recoredsets.update(zone, recordset, data)

Delete

Delete a RecordSet.

client.recoredsets.delete(zone, recordset)

Manager - TLDs

List

List TLDs.

client.tlds.list()

Get

Get a TLD

client.tlds.get(tld)

Create

Create a TLDs.

client.tlds.create(name, data)

Update

Update a TLD.

client.tlds.update(tld, data)

Delete

Delete a TLD.

client.tlds.delete(tld)

Manager - Blacklists

List

List Blacklists.

client.blacklists.list()

Get

Get a Blacklist.

client.blacklists.get(blacklist)

Create

Create a Blacklist.

client.blacklists.create(name, pattern)

Update

Update a Blacklist.

client.blacklists.update(blacklist, pattern)

Delete

Delete a Blacklist.

client.blacklists.delete(blacklist)

Manager - FloatingIPs

List

client.floatingips.list()

Get

client.floatingips.get(region, floatingip_id)

Set

Set a PTR record for a FloatingIP.

client.floatingips.set(region, floatingip_id, record)

Unset

Unset a PTR record for a FloatingIP.

client.floatingips.unset(region, floatingip_id)

Implementation

Assignee(s)

Primary assignee:: endre-karlson

Milestones

Target Milestone for completion:: Juno-2

Work Items

N/A

Dependencies

switch-to-keystone-session

Use Keystone Client session / discovery code

Sun, 06 Jul 2014 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

BP: https://blueprints.launchpad.net/python-designateclient/+spec/switch-to-keystone-session

Problem description

Make the CLI use the Session pattern which should be better then the current approach that’s based on a hook / taking the token from ksclient.auth_token after ksclient.authenticate() is called.

Proposed change

V1 client code shouldn’t change except have added capabilities.

We would end up having the following parameters for the designateclient.v1.Client object:

Name	Description
username	Username (v2/v3)
user_id	User’s ID (v3)
user_domain_id	User’s Domain ID (v3)
user_domain_name	User’s Domain Name (v3)
password	Password (v2/v3)
tenant_name	Tenant Name (v2)
tenant_id	Tenant ID (v2)
project_name	Project Name (v3)
project_id	Project ID (v3)
project_domain_name	Project Domain Name (v3)
project_domain_id	Project Domain ID (v3)
auth_url	Auth URL w/wo auth version in it (It will be discovered by ks.discover if not present)
token	Existing authentication Token (v2/v3)
endpoint_type	Endpoint type (v2/v3)
service_type	Service type (v2/v3)
insecure	Require valid SSL certs
cacert	CA Cert to use
cert	SSL Cert to use

Implementation

Assignee(s)

Primary assignee:: endre-karlson

Milestones

Target Milestone for completion:: Kilo-1

Work Items

N/A

Dependencies

python-keystoneclent v0.11.+

Zone Ownership Transfer Between projects

Mon, 16 Jun 2014 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/zone-migration-between-tenants

For many projects, their organizational needs and structure can change. These changes result in the need to move a zone from one project to another without interruption to domain name resolution.

Problem description

Allow for a zone, and its associated recordsets and records to be moved from one project to another.

This will remove the original owning projects ability to modify the zone.

This could also allow a project to create a sub domain, and transfer the ownership to another project. (ie. the IT team of example.net allows developers to manage records in dev-env.example.net)

We allow both scoped (limited to a defined project), and unscoped (anyone with the ID and access key can accept a request)

Terminology

Term	Meaning
zone	DNS Zone (also refered to as a DNS domain)
project	OpenStack keystone project (also refered to as tenant)
transfer	method of transfering the ownership from one project to another (NOT traditional DNS AFXR zone transfer)

Proposed change

New API Endpoint

Parameter	Description	Required
zone_id	ID of the DNS Zone to be tranfered	Yes
description	Description of the transfer	No
target_project_id	Restrict this share request to the specified project	No - defaults to all projects

POST /v2/zones/<zone-id>/tasks/transfer-requests

This creates a transfer request for a zone owned by the requesting project.

It returns an ID and an access key, which should be communicated out of band to the reciepient (via email / IM etc). They will need both of these to accept the transfer

Important

There should only be one active transfer request per zone at any one point in time

POST /v2/zones/c11ae7e0-f558-11e3-a3ac-0800200c9a66/tasks/transfer-requests HTTP/1.1
Accept: application/json
Content-Type: application/json

{
    "transfer_request":{
        "description":"Transfer to Developers",
        "target_project_id":"88cbc4c7-1dee-40be-804c-ecf86962198c"
    }
}

HTTP/1.1 201 Created
Content-Type: application/json; charset=UTF-8
Location: /v2/zones/c11ae7e0-f558-11e3-a3ac-0800200c9a66/tasks/transfers/cddda8f0-f558-11e3-a3ac-0800200c9a66

{
    "transfer_request":{
        "id":"cddda8f0-f558-11e3-a3ac-0800200c9a66",
        "key":"tSzpOAoUYXhuKDyugHV4",
        "target_project_id":"88cbc4c7-1dee-40be-804c-ecf86962198c",
        "description":"Transfer dev-env.example.net to Developers",
        "status":"PENDING",
        "links":{
            "self" : "/v2/zones/tasks/transfer-requests/cddda8f0-f558-11e3-a3ac-0800200c9a66"
        }
    }
}

This creates the tranfer request, which can be accepted by the recipent as detailed below.

GET /v2/zones/<zone-id>/tasks/transfer-requests

This shows the outstanding transfer FROM the requesting project for this zone.

GET /v2/zones/c11ae7e0-f558-11e3-a3ac-0800200c9a66/tasks/transfer-requests HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "transfer_requests":[
        {
            "transfer_request":{
                "id":"cddda8f0-f558-11e3-a3ac-0800200c9a66",
                "zone_id":"c11ae7e0-f558-11e3-a3ac-0800200c9a66",
                "target_project_id":"88cbc4c7-1dee-40be-804c-ecf86962198c",
                "description":"Transfer to Developers",
                "status":"PENDING",
                "links":{
                        "self" : "/v2/zones/tasks/transfer-requests/cddda8f0-f558-11e3-a3ac-0800200c9a66"
                }
            }
        }
    ]
}

GET /v2/zones/tasks/transfer-requests

This shows the transfer requests for zones currently owned by the CALLING users project

GET /v2/zones/tasks/transfer-requests HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "transfer-requests":[
        {
            "transfer_request":{
                "id":"cddda8f0-f558-11e3-a3ac-0800200c9a66",
                "zone_id":"c11ae7e0-f558-11e3-a3ac-0800200c9a66",
                "target_project_id":"88cbc4c7-1dee-40be-804c-ecf86962198c",
                "description":"Transfer to Developers",
                "status":"PENDING",
                "links":{
                    "self" : "/v2/zones/tasks/transfer-requests/cddda8f0-f558-11e3-a3ac-0800200c9a66"
                }
            }
        },
        {
            "transfer_request":{
                "id":"102e64fb-8fae-4c5e-9246-96a642822e03",
                "zone_id":"05e94130-6356-4687-b5ac-36374f99bf2d",
                "description":"Transfer *.www.example.net to web team",
                "status":"PENDING",
                "links":{
                    "self": "/v2/zones/tasks/transfer-requests/102e64fb-8fae-4c5e-9246-96a642822e03"
                }
            }
        }
    ]
}

GET /v2/zones/tasks/transfer-request/<transfer-id>

This shows the transfer details. If the ‘target_project_id’ was set, this will be restricted to that project only.

The “key” will only be shown to the project that the owns the zone being transfered.

GET /v2/zones/tasks/transfer-requests/cddda8f0-f558-11e3-a3ac-0800200c9a66 HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "transfer_request":{
        "id":"cddda8f0-f558-11e3-a3ac-0800200c9a66",
        "key":"tSzpOAoUYXhuKDyugHV4",
        "target_project_id":"88cbc4c7-1dee-40be-804c-ecf86962198c",
        "zone_id":"c11ae7e0-f558-11e3-a3ac-0800200c9a66",
        "description":"Transfer dev-env.example.net to Developers",
        "status":"PENDING",
        "links":{
            "self": "/v2/zones/tasks/transfer-requests/cddda8f0-f558-11e3-a3ac-0800200c9a66"
        }
    }
}

POST /v2/zones/tasks/transfer-accept/

This is how a transfer gets accepted. The accepting project does a POST request with the key to the tranfer URI. If successful, the API returns a loction header to the user, with the API location of the zone.

POST /v2/zones/tasks/transfer-accept HTTP/1.1
Accept: application/json
Content-Type: application/json

{
    "transfer_accept":{
        "key":"tSzpOAoUYXhuKDyugHV4",
        "transfer_request_id":"cddda8f0-f558-11e3-a3ac-0800200c9a66"
    }
}

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "transfer_accept": {
        "id": "1c914bb0-f580-11e3-a3ac-0800200c9a66",
        "zone_id": "c11ae7e0-f558-11e3-a3ac-0800200c9a66",
        "status": "COMPLETE",
        "links": {
            "self": "/v2/zones/tasks/transfer-accept/1c914bb0-f580-11e3-a3ac-0800200c9a66",
            "zone": "/v2/zones/c11ae7e0-f558-11e3-a3ac-0800200c9a66"
    }
}

A further GET on the transfer-request entity will show the following

GET /v2/zones/tasks/transfer-request/<transfer-id> HTTP/1.1
Accept: application/json

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8

{
    "transfer_request":{
        "id":"cddda8f0-f558-11e3-a3ac-0800200c9a66",
        "key":"tSzpOAoUYXhuKDyugHV4",
        "target_project_id":"88cbc4c7-1dee-40be-804c-ecf86962198c",
        "zone_id":"c11ae7e0-f558-11e3-a3ac-0800200c9a66",
        "description":"Transfer dev-env.example.net to Developers",
        "status":"COMPLETE",
        "links":{
            "self": "/v2/zones/<zone-id>/tasks/transfer-requests/cddda8f0-f558-11e3-a3ac-0800200c9a66"
        }
    }
}

DELETE /v2/zones/<zone-id>/tasks/transfer-requests/<request-id>

This allows a user to cancel a tranfer request. This can only be performed before the tranfer has happened, and DOES NOT cause a tranfer to roll back.

DELETE /v2/zones/tasks/transfer-requests/cddda8f0-f558-11e3-a3ac-0800200c9a66 HTTP/1.1

HTTP/1.1 204 No Content

Central Changes

New Central methods to allow for the CRUD of Transfer Requests. The Central methods should use currently in place update method for changing the project ownership of the referenced zone

Storage Changes

New Table - ZoneTransferRequestTasks

Row	Type	Nullable?	Unique?	Notes
id	uuid	No	Yes
zone_id	uuid	No	Yes
key	VARCHAR	No	No
description	VARCHAR	Yes	No
project_id	uuid	No	No	The project that created the ZoneTransferRequest
target_project_id	uuid	Yes	No
status	ENUM(PENDING,COMPLETE,DELETED,ERROR)	No	No

New Table - ZoneTransferAcceptTasks

Row	Type	Nullable?	Unique?	Notes
id	uuid	No	Yes
zone_id	uuid	No	Yes
project_id	uuid	No	No	The ID of the project that accepted the tranfer request
transfer_request_id	uuid	No	Yes
status	ENUM(COMPLETE,ERROR)	No	No

Implementation

Assignee(s)

Primary assignee:: https://launchpad.net/~grahamhayes

Milestones

Target Milestone for completion:: Juno-2

Work Items

Tasks Sub API
CRUD for the two tasks (Request & Accept)
Moving Zone to new tenant

Dependencies

None

References

Server Pools

Mon, 16 Jun 2014 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/designate/+spec/server-pools

Problem description

Server Pools are required for a few different scenarios:

Private Pools

This allows users to have ‘private’ DNS servers. These servers would typically allow non standard TLDs (.dev , .local etc), and may not have the same level of blacklist restrictions. They would be aimed at people with Neutron Networking, and VPC style set ups, where access to the DNS server would come from trusted networks (E.G. in-cloud - owned instances, and onsite resources connect by VPN)

This would allow customers to set DNS entries for internal servers, on domains that would not be available on the public pools, and have them accessible by internal users

Distribution

Having multiple public pools with the same capabilities, would allow the scheduler to distribute zones across multiple infrastructures.

As part of the pools change we are also changing how the servers API works, to allow for more fine grained control of servers, their capabilities and backends

Features / Premium Systems

By using scheduler hints, we can mark different pools as having different capabilities - such as GeoIP / Round Robin DNS / Anycast.

This allows operators to run different DNS infrastructures as required. For example this allow users to have some zones on pools with GeoIP, and pay a premium for this feature, while having the rest of their zones on a cheaper ‘standard’ tier.

Proposed change

Terminology

The terms for different parts of pool manager can be confusing, so the following list should be use when referring to components of Server Pools

Term	Meaning	Also known as
Server Pool	group of DNS servers
DNS Zone	DNS Zone (aka subzone.domain.com.) more commonly referred to as a domain
Zone	See DNS Zone
Presentation DNS Server	Customer Facing DNS server (used by people to resolve zones owned by designate)
Name Server	A FQDN (or IP, but usually a FQDN) that is used to populate the NS Records of a Designate Managed Zone. Each Pool will have a set of Name Servers, which users then delegate to from their registrar	nameserver, ns record
Backend	A driver that allows Designate to control a particular type of DNS Server software (BIND, PowerDNS etc)
MiniDNS	A designate service that is used to send notifies to servers that need to be updated (with new information about Zones), and serves AFXR requests with the new information. There is usually a shared set of MiniDNS servers for all pools
FQDN	Fully Qualified Domain Name - a DNS entry that has both a hostname section (i.e. ns1. ) and a zone section (including the trailing ‘.’) (i.e. example.com. )

This will allow us to split domains between different groups of servers. This will have a fairly massive impact across the whole system - and will require changes to nearly every part of Designate.

We will add 2 major pieces to Designate:

Pool Manager

This is a service that will be responsible for notifying DNS servers that changes have occured. This will take the backends implemented for MiniDNS, and load them.

It will take the place of the miniDNS notifier section, and will be responsible for checking async operations.

Scheduler

This service will be resonsible for assigning zones to pools.

The scheduler will assign a zone to a pool based on hints in the request, or if there is no hints in the request, any pool that has capacity.

Initially for pools there a single ‘default pool’ defined in the config, and overrides will be allowed via the ‘hints’ section of a zone create API request

Flow of information with Server Pools

Single Frames available [1]

Overview Image

https://wiki.openstack.org/w/images/a/a7/Designate-MiniDNS-Pools.gif [2]

We have discusses this in person twice now - once in the Icehouse mid-cycle, and once in Atlanta at the design summit.

Etherpad

Milestones

Target Milestone for completion:: Kilo-2 [3]

Work Items

Work Item	Assignee	Priority
server-pools-storage	https://launchpad.net/~rjrjr / https://launchpad.net/~darshan104	High
server-pool-manager	https://launchpad.net/~rjrjr / https://launchpad.net/~darshan104	High
server-pools-minidns-support	https://launchpad.net/~vinod-mang	High
server-pools-api	https://launchpad.net/~betsy-luzader	High

Dependencies

MiniDNS