Subject: [quicwg/base-drafts] Add more context and text around ACK frequency (#3706)

From: Jana Iyengar <[email-masked]>
Date: Wed, 27 May 2020 20:05:55 -0700

This PR adds explicit context around the choice of ACK frequency in the draft. It also takes in Bob Briscoe&#39;s suggestion in #3529, of calling it a guidance in light of the best information we have.

Also does some editorial cleanup of text blocks that were not in the right sections.
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/3706

-- Commit Summary --

  * Add more context and text around ACK frequency

-- File Changes --

    M draft-ietf-quic-transport.md (85)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/3706.patch
https://github.com/quicwg/base-drafts/pull/3706.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706

---

From: Martin Thomson <[email-masked]>
Date: Wed, 27 May 2020 20:57:51 -0700

@martinthomson commented on this pull request.

This is a little too strong in its current form, I think.

> @@ -3413,17 +3421,10 @@ Similarly, packets marked with the ECN Congestion Experienced (CE) codepoint in
 the IP header SHOULD be acknowledged immediately, to reduce the peer's response
 time to congestion events.
 
-As an optimization, a receiver MAY process multiple packets before sending any
-ACK frames in response.  In this case the receiver can determine whether an
-immediate or delayed acknowledgement should be generated after processing
-incoming packets.
-
-Packets containing PADDING frames are considered to be in flight for congestion
-control purposes {{QUIC-RECOVERY}}. Sending only PADDING frames might cause the
-sender to become limited by the congestion controller with no acknowledgments
-forthcoming from the receiver. Therefore, a sender SHOULD ensure that other
-frames are sent in addition to PADDING frames to elicit acknowledgments from
-the receiver.
+The algorithms in {{QUIC-RECOVERY}} are resilient to receivers that do not

do we want to add "generally" to "resilient"?  Or, "in principle"?  Or "in theory"?

> +which could prevent the connection from ever becoming idle. Note that
+non-ack-eliciting packets are eventually acknowledged when the endpoint sends an
+ACK frame in response to other events.

```suggestion
which could prevent the connection from ever becoming idle.
Non-ack-eliciting packets are eventually acknowledged when the endpoint sends an
ACK frame in response to other events.
```

> +A receiver determines how frequently to send acknowledgements in response to
+ack-eliciting packets. Under normal circumstances, reducing the frequency of
+acknowledgement packets can improve connection and endpoint performance in
+several ways, such as reducing computational cost at both endpoints, and
+improving connection throughput on severely asymmetric links.

You need to phrase this more as a trade-off.  This is concentrated on the benefits of reducing acknowledgments.  But the introduction here also needs to recognize that generating acknowledgments in a timely fashion is critical for performance.

> -one ACK-frame-only packet in response to receiving an ack-eliciting packet.  An
-endpoint MUST NOT send a non-ack-eliciting packet in response to a
-non-ack-eliciting packet, even if there are packet gaps which precede the
-received packet. Limiting ACK frames avoids an infinite feedback loop of
-acknowledgements, which could prevent the connection from ever becoming idle.
-However, the endpoint acknowledges non-ACK-eliciting packets when it sends an
-ACK frame.
+### Acknowledgement Frequency
+
+A receiver determines how frequently to send acknowledgements in response to
+ack-eliciting packets. Under normal circumstances, reducing the frequency of
+acknowledgement packets can improve connection and endpoint performance in
+several ways, such as reducing computational cost at both endpoints, and
+improving connection throughput on severely asymmetric links.
+
+However, there are undesirable consequences to a receiver simply reducing the

If you balance the first paragraph out, then you don't need a "however" here.  You can just talk in more detail about the advantages of more frequent acknowledgment.

> +acknowledgement frequency, especially to an arbitrary fixed value. A sender
+relies on receipt of acknowledgements to determine the amount of data in flight
+and to detect loss; see {{QUIC-RECOVERY}}. Loss detection can be delayed with
+late acknowledgements. Window-based congestion controllers, such as the one
+defined in {{QUIC-RECOVERY}}, rely on receipt of acknowledgments to increase
+their sending rate. Consequently, a connection can suffer performance penalties

If you move the general statement at the end of this to the first paragraph, then you can probably shorten this paragraph to: 

> Endpoints rely on timely acknowledgment to detect loss; see Section X of {{QUIC-RECOVERY}}.  Window-based congestion controllers, such as the one in Section X of {{QUIC-RECOVERY}} rely on acknowledgments to manage their sending rate.  In both cases, delaying acknowledgment can adversely affect performance.

We don't need the specific dig regarding the "arbitrary fixed value".

> +Further research and experimentation will yield effective mechanisms to balance
+this tradeoff. Meanwhile, this document provides the following guidance:
+{{sending-acknowledgements}}, a receiver SHOULD generate an ACK frame for at
+least every second ack-eliciting packet. This recommendation is in keeping with

A receiver SHOULD send an ACK frame after receiving at least two ack-eliciting packets.  Better knowledge of network conditions or further research and experimentation might suggest alternative acknowledgment strategies with better performance characteristics.  This recommendation is general in nature and consistent with recommendations for TCP endpoint behavior {{?RFC5681}}.

> @@ -3555,6 +3566,16 @@ messages are delayed or lost.  Note that the same limitation applies to other
 data sent by the server protected by the 1-RTT keys.

+### Other Considerations

I'm sure that we can do better than "Other Considerations" here.

```suggestion
### PADDING Frames Consume Congestion Window
```

> @@ -3555,6 +3566,16 @@ messages are delayed or lost.  Note that the same limitation applies to other
 data sent by the server protected by the 1-RTT keys.

+### Other Considerations
+
+Packets containing PADDING frames are considered to be in flight for congestion
+control purposes {{QUIC-RECOVERY}}. Sending only PADDING frames might cause the
+sender to become limited by the congestion controller with no acknowledgments
+forthcoming from the receiver. Therefore, a sender SHOULD ensure that other
+frames are sent in addition to PADDING frames to elicit acknowledgments from

```suggestion
frames are sent periodically in addition to PADDING frames to elicit acknowledgments from
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#pullrequestreview-419755966

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 28 May 2020 16:27:54 -0700

@janaiyengar pushed 1 commit.

2f2fc2a6cf3f6cb45a99a795754d2f740163db43  Apply suggestions from code review

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706/files/f1fb3b57b37f0ee1dd366e46f992c48fb5d51db6..2f2fc2a6cf3f6cb45a99a795754d2f740163db43

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 28 May 2020 16:46:55 -0700

9e6d2f14f14f6ffc50d6c31d4143223202411e4c  mt's excellent suggestions

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706/files/2f2fc2a6cf3f6cb45a99a795754d2f740163db43..9e6d2f14f14f6ffc50d6c31d4143223202411e4c

---

From: Martin Thomson <[email-masked]>
Date: Thu, 28 May 2020 16:53:30 -0700

@martinthomson approved this pull request.

Thanks!  This appears to be much better.  And it points in the right direction without being overly prescriptive.

-Packets containing only ACK frames are not congestion controlled, so there are
-limits on how frequently they can be sent.  An endpoint MUST NOT send more than
-one ACK-frame-only packet in response to receiving an ack-eliciting packet.  An
-endpoint MUST NOT send a non-ack-eliciting packet in response to a
-non-ack-eliciting packet, even if there are packet gaps which precede the
-received packet. Limiting ACK frames avoids an infinite feedback loop of
-acknowledgements, which could prevent the connection from ever becoming idle.
-However, the endpoint acknowledges non-ACK-eliciting packets when it sends an
-ACK frame.
+### Acknowledgement Frequency
+
+A receiver determines how frequently to send acknowledgements in response to
+ack-eliciting packets. This determination involves a tradeoff however.

```suggestion
ack-eliciting packets. This determination involves a tradeoff.
```

-Packets containing only ACK frames are not congestion controlled, so there are
-limits on how frequently they can be sent.  An endpoint MUST NOT send more than
-one ACK-frame-only packet in response to receiving an ack-eliciting packet.  An
-endpoint MUST NOT send a non-ack-eliciting packet in response to a
-non-ack-eliciting packet, even if there are packet gaps which precede the
-received packet. Limiting ACK frames avoids an infinite feedback loop of
-acknowledgements, which could prevent the connection from ever becoming idle.
-However, the endpoint acknowledges non-ACK-eliciting packets when it sends an
-ACK frame.
+### Acknowledgement Frequency
+
+A receiver determines how frequently to send acknowledgements in response to
+ack-eliciting packets. This determination involves a tradeoff however.
+
+Endpoints rely on timely acknowledgment to detect loss; see Section X of

We'll need a value for X.

```suggestion
Endpoints rely on timely acknowledgment to detect loss; see Section 5 of
```

> -limits on how frequently they can be sent.  An endpoint MUST NOT send more than
-one ACK-frame-only packet in response to receiving an ack-eliciting packet.  An
-endpoint MUST NOT send a non-ack-eliciting packet in response to a
-non-ack-eliciting packet, even if there are packet gaps which precede the
-received packet. Limiting ACK frames avoids an infinite feedback loop of
-acknowledgements, which could prevent the connection from ever becoming idle.
-However, the endpoint acknowledges non-ACK-eliciting packets when it sends an
-ACK frame.
+### Acknowledgement Frequency
+
+A receiver determines how frequently to send acknowledgements in response to
+ack-eliciting packets. This determination involves a tradeoff however.
+
+Endpoints rely on timely acknowledgment to detect loss; see Section X of
+{{QUIC-RECOVERY}}. Window-based congestion controllers, such as the one in
+Section X of {{QUIC-RECOVERY}}, rely on acknowledgments to manage their sending

```suggestion
Section 6 of {{QUIC-RECOVERY}}, rely on acknowledgments to manage their sending
```

> +packets. Better knowledge of network conditions or further research and
+experimentation might suggest alternative acknowledgment strategies with better
+performance characteristics. This recommendation is general in nature and
+consistent with recommendations for TCP endpoint behavior {{?RFC5681}}.

I think maybe we can swap the last two sentences, so that "this recommendation" is better bound to the first sentence.

> @@ -3555,6 +3563,16 @@ messages are delayed or lost.  Note that the same limitation applies to other

+### PADDING Frames Consume Congestion Window
+
+Packets containing PADDING frames are considered to be in flight for congestion
+control purposes {{QUIC-RECOVERY}}. Sending only PADDING frames might cause the
+sender to become limited by the congestion controller with no acknowledgments
+forthcoming from the receiver. Therefore, a sender SHOULD ensure that other
+frames are sent periodically in addition to PADDING frames to elicit acknowledgments from

Check line length.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#pullrequestreview-420566327

---

From: ianswett <[email-masked]>
Date: Thu, 28 May 2020 18:13:50 -0700

@ianswett commented on this pull request.

This seems to be heading in a good direction, thanks for working on it.

> +Section X of {{QUIC-RECOVERY}}, rely on acknowledgments to manage their sending
+rate. In both cases, delaying acknowledgment can adversely affect performance.

```suggestion
Section X of {{QUIC-RECOVERY}}, rely on acknowledgments to open up congestion window,
allowing more packets to be sent, as well as increase the congestion window. In both
cases, delaying acknowledgments can adversely affect performance.
```

+rate. In both cases, delaying acknowledgment can adversely affect performance.
+
+On the other hand, reducing the frequency of acknowledgement packets reduces

Do you mean ACK-only packets?  Otherwise this could be read to include ACK frames bundled with data, which I don't believe is the intent.

+{{QUIC-RECOVERY}}. Window-based congestion controllers, such as the one in
+Section X of {{QUIC-RECOVERY}}, rely on acknowledgments to manage their sending
+rate. In both cases, delaying acknowledgment can adversely affect performance.
+
+On the other hand, reducing the frequency of acknowledgement packets reduces
+packet processing cost at both endpoints. It can also improve connection
+throughput on severely asymmetric links; see Section 3 of {{?RFC3449}}.
+
+A receiver SHOULD send an ACK frame after receiving at least two ack-eliciting
+packets. Better knowledge of network conditions or further research and
+experimentation might suggest alternative acknowledgment strategies with better
+performance characteristics. This recommendation is general in nature and
+consistent with recommendations for TCP endpoint behavior {{?RFC5681}}.
+
+A receiver MAY process multiple packets before sending any ACK frames in

```suggestion
A receiver MAY process multiple available packets before determining whether to sending an ACK frame in
```

> +response. This allows for a receiver to process packets already queued for
+processing before determining whether to send an acknowledgement.

```suggestion
response.
```

```suggestion
packets. Knowledge of how frequently the peer's congestion controller needs feedback,
network conditions or further research and experimentation might suggest alternative
acknowledgment strategies with better performance characteristics. This
recommendation is general in nature and
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#pullrequestreview-420588453

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 28 May 2020 22:41:04 -0700

@janaiyengar commented on this pull request.

Gah! Sorry, meant to replace that.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#discussion_r432266003

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 28 May 2020 22:43:43 -0700

f1d62609d466ca019f95677c6fdb5396544a7812  Apply suggestions from code review

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706/files/9e6d2f14f14f6ffc50d6c31d4143223202411e4c..f1d62609d466ca019f95677c6fdb5396544a7812

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 28 May 2020 22:54:05 -0700

e34dcf87d36270b9fd685c683a0c2fcfdec3a44d  ian/mt comments

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706/files/f1d62609d466ca019f95677c6fdb5396544a7812..e34dcf87d36270b9fd685c683a0c2fcfdec3a44d

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 28 May 2020 22:54:43 -0700

Thanks @ianswett @martinthomson -- comments incorporated.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#issuecomment-635775298

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 29 May 2020 00:49:14 -0700

@kazuho approved this pull request.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#pullrequestreview-420726488

---

From: Martin Thomson <[email-masked]>
Date: Sun, 31 May 2020 16:53:02 -0700

+sender to become limited by the congestion controller with no acknowledgments
+forthcoming from the receiver. Therefore, a sender SHOULD ensure that other
+frames are sent periodically in addition to PADDING frames to elicit
+acknowledgments from the receiver.

```suggestion
control purposes {{QUIC-RECOVERY}}. Packets containing only PADDING frames
therefore consume congestion window but do not necessarily generate
acknowledgments that will open the congestion window. Therefore, a sender SHOULD
ensure that other frames are sent periodically in addition to PADDING frames to
elicit acknowledgments from the receiver.
```

A maybe-improvement.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#pullrequestreview-421539720

---

From: ianswett <[email-masked]>
Date: Sun, 31 May 2020 17:57:11 -0700

-one ACK-frame-only packet in response to receiving an ack-eliciting packet.  An
-endpoint MUST NOT send a non-ack-eliciting packet in response to a
-non-ack-eliciting packet, even if there are packet gaps which precede the
-received packet. Limiting ACK frames avoids an infinite feedback loop of
-acknowledgements, which could prevent the connection from ever becoming idle.
-However, the endpoint acknowledges non-ACK-eliciting packets when it sends an
-ACK frame.
+### Acknowledgement Frequency
+
+A receiver determines how frequently to send acknowledgements in response to
+ack-eliciting packets. This determination involves a tradeoff.
+
+Endpoints rely on timely acknowledgment to detect loss; see Section 5 of
+{{QUIC-RECOVERY}}. Window-based congestion controllers, such as the one in
+Section 6 of {{QUIC-RECOVERY}}, rely on acknowledgments to manage their sending

I would still like to be clearer here and I don't really like "sending rate" in the context of window-based controllers.  I made some specific suggestions before.  Can you take another look at them?

-endpoint MUST NOT send a non-ack-eliciting packet in response to a
-non-ack-eliciting packet, even if there are packet gaps which precede the
-received packet. Limiting ACK frames avoids an infinite feedback loop of
-acknowledgements, which could prevent the connection from ever becoming idle.
-However, the endpoint acknowledges non-ACK-eliciting packets when it sends an
-ACK frame.
+### Acknowledgement Frequency
+
+A receiver determines how frequently to send acknowledgements in response to
+ack-eliciting packets. This determination involves a tradeoff.
+
+Endpoints rely on timely acknowledgment to detect loss; see Section 5 of
+{{QUIC-RECOVERY}}. Window-based congestion controllers, such as the one in
+Section 6 of {{QUIC-RECOVERY}}, rely on acknowledgments to manage their sending
+rate. In both cases, delaying acknowledgment can adversely affect performance.

```suggestion
rate. In both cases, delaying acknowledgments can adversely affect performance.
```

-acknowledgements, which could prevent the connection from ever becoming idle.
-However, the endpoint acknowledges non-ACK-eliciting packets when it sends an
-ACK frame.
+### Acknowledgement Frequency
+
+A receiver determines how frequently to send acknowledgements in response to
+ack-eliciting packets. This determination involves a tradeoff.
+
+Endpoints rely on timely acknowledgment to detect loss; see Section 5 of
+{{QUIC-RECOVERY}}. Window-based congestion controllers, such as the one in
+Section 6 of {{QUIC-RECOVERY}}, rely on acknowledgments to manage their sending
+rate. In both cases, delaying acknowledgment can adversely affect performance.
+
+On the other hand, reducing the frequency of packets that carrying only
+acknowledgements reduces packet processing cost at both endpoints. It can also

Q: Doesn't it reduce packet serialization and sending costs at the receiver and packet processing costs at the sender?  In practice, the cost of sending ACK-only packets is commonly worse, so it seems worth mentioning.

I'd remove "necessarily" from your suggestion, but otherwise LGTM.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#pullrequestreview-421546265

---

From: Martin Thomson <[email-masked]>
Date: Sun, 31 May 2020 19:00:48 -0700

Suggestion updated.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#discussion_r433013233

---

From: Jana Iyengar <[email-masked]>
Date: Sun, 31 May 2020 19:32:04 -0700

That is a detail, which will need further substantiation. The point here is to note _some_ benefits, to point out that there are some. The specifics will depend on your particular platform.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#discussion_r433018199

---

From: Jana Iyengar <[email-masked]>
Date: Sun, 31 May 2020 19:33:05 -0700

12b886d9ccc4cc16b019d9e91ed15abd83dbb456  Apply suggestions from code review

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706/files/e34dcf87d36270b9fd685c683a0c2fcfdec3a44d..12b886d9ccc4cc16b019d9e91ed15abd83dbb456

---

From: Jana Iyengar <[email-masked]>
Date: Sun, 31 May 2020 19:35:07 -0700

It's still quite accurate, IMO. Ultimately, increasing the congestion window increases the sending rate. That is the higher order point here -- that reducing ack frequency reduces the throughput of the sender.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#discussion_r433018697

---

From: Jana Iyengar <[email-masked]>
Date: Sun, 31 May 2020 19:37:24 -0700

I've changed it to "manage their congestion window"

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#discussion_r433019079

---

From: Jana Iyengar <[email-masked]>
Date: Sun, 31 May 2020 19:37:53 -0700

3c9f34272c60e7bd6607197bec94fcd889df83bc  s/sending rate/congestion window

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706/files/12b886d9ccc4cc16b019d9e91ed15abd83dbb456..3c9f34272c60e7bd6607197bec94fcd889df83bc

---

From: Marten Seemann <[email-masked]>
Date: Sun, 31 May 2020 20:06:29 -0700

@marten-seemann approved this pull request.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#pullrequestreview-421571073

---

From: ianswett <[email-masked]>
Date: Sun, 31 May 2020 21:46:42 -0700

Now that I read this again, I'm actually not sure what you mean.  Do you mean packet processing cost as in the cost of receiving a large number of data packets, so fewer ACK-only packets reduces that cost?

I was thinking you were saying it reduces the number of ACKs that have to be received and processed.

Either way, I have a small suggestion.
```suggestion
acknowledgements reduces packet transmission and processing cost at both endpoints. It can also
```

+Section 6 of {{QUIC-RECOVERY}}, rely on acknowledgments to manage their
+congestion window. In both cases, delaying acknowledgments can adversely affect
+performance.

```suggestion
On the other hand, reducing the frequency of packets that carry only
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#pullrequestreview-421589751

---

From: ianswett <[email-masked]>
Date: Sun, 31 May 2020 21:50:08 -0700

Thanks, I think that's better, but I also think it doesn't highlight a key point, which is if a receiver delays sending an ACK for Xms, the sender may be blocked from sending for Xms.

Except for slow start, I don't think the congestion window itself is the issue, but rather the available congestion window.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706#discussion_r433039899

---

From: Jana Iyengar <[email-masked]>
Date: Sun, 31 May 2020 23:25:00 -0700

76af223d868f9d50662e00245cb2e167e25baec8  Apply suggestions from code review

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706/files/3c9f34272c60e7bd6607197bec94fcd889df83bc..76af223d868f9d50662e00245cb2e167e25baec8

---

From: Jana Iyengar <[email-masked]>
Date: Sun, 31 May 2020 23:26:34 -0700

3bbb568d4860754976861809e9f313d5aaeb83d8  reflow

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3706/files/76af223d868f9d50662e00245cb2e167e25baec8..3bbb568d4860754976861809e9f313d5aaeb83d8