buglwIP - A Lightweight TCP/IP stack - Bugs: bug #64010, oss-fuzz integration

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #64010: oss-fuzz integration

Submitter:  J. Neuschäfer <jne>
Submitted:  Thu 06 Apr 2023 03:25:39 PM UTC
   
 
Category:  Security-related Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Planned Release:  None
lwIP version:  git head

Jump to the original submission

Tue 18 Apr 2023 11:21:23 AM UTC, comment #14: 


> Otoh, we (as lwIP) do not even provide our sources as GPL but "only" as BSD,..


"BSD" is ambiguous: multiple software licenses have "BSD" in their name; all widely-known ones are free software licenses, but some of them are GPL-incompatible, therefore unacceptable for Savannah.

> so we're probably part of the "bad guys" as well?


Definitely not; if the GNU Project thought so, we wouldn't list permissive licenses in our registration form.

> And finally, let's face reality: we do have a github mirror as well.


As I said in comment #10, mirroring a repository isn't a SaaSS; even if it's problematic on GitHub, the problems are different.

> We even use it for our CI. And we do so because we're lazy and github "just works". Does that make us even more bad?


It depends on what your CI does.  SaaSS, like proprietary software, primarily harms its users, so "bad" isn't the most exact word.  However, I think you shouldn't use Savannah to direct other people to such things.

Ineiev <ineiev>
Fri 14 Apr 2023 06:11:13 PM UTC, comment #13: 

To come back to this issue from a project leader point of view, I'm now not sure what to say.

I can repeat what I said in comment #3 ("We do have fuzzing, only we don't have the computing power to extensively run it..."), but I guess for the "hardcore" GNU guys, that doesn't count?

Otoh, we (as lwIP) do not even provide our sources as GPL but "only" as BSD, so we're probably part of the "bad guys" as well?

And finally, let's face reality: we do have a github mirror as well. We even use it for our CI. And we do so because we're lazy and github "just works". Does that make us even more bad?

That being said, I'm thankful for jne's work on fuzzing, but I'm still not sure oss-fuzz is a good idea, both because of the mentioned "free XYZ" issues (the question for me here is "why do we get that service for free?") and because I'm afraid to get too many reports I have to handle. However, I'd still be willing to try some kind of "fuzzing-CI".

Simon Goldschmidt <goldsimon>
Group administrator
Fri 14 Apr 2023 08:29:53 AM UTC, comment #12: 

comment #11:

> You can define your own Dockerfile and define your container for your project and in theory, do aerial imagery processing in it. On the other hand, it will probably not be accepted if it is unrelated to fuzzing.


If the user (other than Google itself) has to rely on Google's acceptance in order to use it, if the user has to take Google's word about what really happens inside the instance they use, then we can't say that the user is free.

> It is similar to e.g. [2].


It isn't clear whether the analogy really holds (e.g. coreutils developers may have dissimilar relations with service maintainers) and if yes, whether it's a bug that should be fixed.  It's a different issue, it should be discussed separately.

> So a useful service, so open and still on the wrong side because it is clearly SaaSS by this definition.


The service is on the wrong side because it disrespects its users' freedom, so the definition does a good job in this case.

> Because back in the days when CI was not a thing, someone told us that SaaSS was evil.
> So we'll stay in IT prehistory just for that.


I believe the novelty is exaggerated.  Freedom has been being taken away for millennia, and proprietary software has been positioned as a useful thing since its very early days, otherwise people wouldn't have bought it.

> Should we add artificial backdoors to build containers to fulfill the idea that we won't be considered SaaSS? That seems to be the only difference.


Why do you say "we"?  Do you speak for Google?

> So if we would like to use OSS-Fuzz, we should create an unofficial mirror, do it there, and don't tell you about it ;-)


The main point is: you shouldn't use the resources of the GNU Project to legitimize services that deny its users freedom.

Ineiev <ineiev>
Thu 13 Apr 2023 10:14:01 AM UTC, comment #11: 

comment #10:

> Do the users have a superuser access to their container? Can they modify their OSS-Fuzz instance in it?  Can they replace or combine their instance with something unrelated they want to use, like aerial imagery processing?
>


You can define your own Dockerfile and define your container for your project and in theory, do aerial imagery processing in it. On the other hand, it will probably not be accepted if it is unrelated to fuzzing.

It is similar to e.g. [2]. They are doing continuous integration tasks for coreutils. Coreutils maintainers can define anything that should be done. The service is open-source (GPL-3).
But if coreutils maintainers decide to run aerial imagery processing during the build, Hydra maintainers will probably not be happy. It probably wouldn't even pass the coreutils maintener review.
They also probably don't have root access to the container while it is running.
So a useful service, so open and still on the wrong side because it is clearly SaaSS by this definition.

I understand your views, but it still makes me sad.
Because back in the days when CI was not a thing, someone told us that SaaSS was evil.
So we'll stay in IT prehistory just for that.
Should we add artificial backdoors to build containers to fulfill the idea that we won't be considered SaaSS? That seems to be the only difference.

> First, it's questionable whether that mirror is really official. 


See [1] "The latest source with revision history can be browsed using cgit, gitweb or GitHub."

> I couldn't find any references of OSS-Fuzz in Coreutils; since it's free software, its users may do whatever they wish with it, including submitting it to third-party services.


So if we would like to use OSS-Fuzz, we should create an unofficial mirror, do it there, and don't tell you about it ;-)



[1] https://www.gnu.org/software/coreutils/#source
[2] https://hydra.nixos.org/jobset/gnu/coreutils-master

Jan Breuer <jan_breuer>
Thu 13 Apr 2023 08:10:09 AM UTC, comment #10: 


> According to the requirements [1], the use of non-free software is prohibited. Even just talking about it here is forbidden.


This isn't correct.  The requirement is to work on free platforms better or as well as on nonfree ones.   It doesn't go as long as prohibiting support for proprietary software, to say nothing of using it.  Proprietary software mistreats its users, that's why they shouldn't use it.

> RMS tries to specify the boundary between SaaSS and renting (virtual) hardware [3] and he does not consider renting virtual hardware as bad.


He says that without prejudice.  Virtual hardware renting may turn out wrong for other reasons.

> But, unless you own the whole datacenter, the line between proprietary SaaSS and renting virtual hardware is so blurry.


The free vs. proprietary distinction doesn't apply to services.

> Is renting a virtual server with a virtual disk in proprietary infrastructure ok? Is renting a container with blob storage in proprietary infrastructure ok?


Likewise, the GNU Project doesn't maintain the concept of proprietary infrastructure.  You may introduce it yourself, but you shouldn't make any assumptions about approval or disapproval of such things by the GNU Project.

> The reason why am I talking about it is, that it is exactly what is happening here. Free software (OSS-Fuzz) is running in a free software container in a proprietary container manager (Cloud Build [8]) and proprietary infrastructure behind it.


Do the users have a superuser access to their container? Can they modify their OSS-Fuzz instance in it?  Can they replace or combine their instance with something unrelated they want to use, like aerial imagery processing?

If not, the service is certainly on the wrong side, no matter how blurry the border is.

> Here is also at least one precedent: GNU Coreutils.


Such precedents are not binding for our purpose.  People make mistakes, and GNU maintainers are no exception.

>  - Official repository of one of the core GNU projects [5]
>  - Official GitHub (nonfree service) mirror [6]


First, it's questionable whether that mirror is really official.  Coreutils only mention it as a way to browse its sources, I failed to find an explicit statement that it's an official mirror.

Then, mirroring a repository is definitely not a SaaSS, even though GitHub is well-known for its hostility to free software.

>  - OSS-Fuzz usage [7]


I couldn't find any references of OSS-Fuzz in Coreutils; since it's free software, its users may do whatever they wish with it, including submitting it to third-party services.

Ineiev <ineiev>
Tue 11 Apr 2023 08:28:31 AM UTC, comment #9: 

According to the requirements [1], the use of non-free software is prohibited. Even just talking about it here is forbidden.

On the other hand, oss-fuzz itself is free software covered by Apache-2.0 [2] license. It defines the whole infrastructure, containers, and how to run them. All used fuzzers are also free software covered by Apache-2.0 or more free licenses (BSD-3-Clause, ...), all compatible with GPL-3.0+ meeting all requirements.

RMS tries to specify the boundary between SaaSS and renting (virtual) hardware [3] and he does not consider renting virtual hardware as bad. But, unless you own the whole datacenter, the line between proprietary SaaSS and renting virtual hardware is so blurry. Is renting a virtual server with a virtual disk in proprietary infrastructure ok? Is renting a container with blob storage in proprietary infrastructure ok?

The reason why am I talking about it is, that it is exactly what is happening here. Free software (OSS-Fuzz) is running in a free software container in a proprietary container manager (Cloud Build [8]) and proprietary infrastructure behind it. Results are stored in blob storage (GCS [4]) which we should consider being just storage of data and not SasSS [9] because it does not perform further processing.

Here is also at least one precedent: GNU Coreutils.

 - Official repository of one of the core GNU projects [5]
 - Official GitHub (nonfree service) mirror [6]
 - OSS-Fuzz usage [7]

Was this a big mistake or a practical decision and modernization?


[1]: https://savannah.nongnu.org/register/requirements.php
[2]: https://github.com/google/oss-fuzz/blob/master/LICENSE
[3]: https://www.gnu.org/philosophy/who-does-that-server-really-serve.html#renting
[4]: https://cloud.google.com/storage
[5]: https://savannah.gnu.org/projects/coreutils
[6]: https://github.com/coreutils/coreutils
[7]: https://github.com/google/oss-fuzz/blob/master/projects/coreutils/project.yaml
[8]: https://cloud.google.com/build
[9]: https://www.gnu.org/philosophy/who-does-that-server-really-serve.html section Distinguishing SaaSS from Other Network Services

Jan Breuer <jan_breuer>
Sat 08 Apr 2023 06:12:16 PM UTC, comment #8: 

Thanks for reading the article.

comment #7:

> Or, to take the "Service as a Software Substitute" phrase literally:
>
> No, it doesn't apply very well in this case. What OSS-Fuzz substitutes is running afl-fuzz on one's own hardware.


Er... yes, this is what SaaSS about, and this is what we recommend against.  People should use hardware they fully control for their computations.

> OSS-Fuzz is an unpaid service as a paid-service substitute.


It doesn't matter if it's paid or unpaid.  What does matter is that it denies freedom.  Google won't let the users modify OSS-Fuzz as they wish, and even check if what Google runs is really afl-fuzz.  In order to do anything like that, the users have to arrange and run all themselves.

Ineiev <ineiev>
Sat 08 Apr 2023 05:42:04 PM UTC, comment #7: 

Or, to take the "Service as a Software Substitute" phrase literally:

No, it doesn't apply very well in this case. What OSS-Fuzz substitutes is running afl-fuzz on one's own hardware. The novelty of OSS-Fuzz is not in the software (afl-fuzz is perfectly available, and any improvements in lwIP's fuzzing wrappers are useful with and without OSS-Fuzz), but in the service of providing hardware and power for running fuzzers.

OSS-Fuzz is an unpaid service as a paid-service substitute.

J. Neuschäfer <jne>
Sat 08 Apr 2023 05:28:29 PM UTC, comment #6: 

It is true, OSS-Fuzz is SaaS (and inconveniently requires using a Google account), and thus may be undesirable for some people.

However, I think in the case of OSS-Fuzz, the dependence and vendor lock-in is limited, since most of the interesting parts exist outside of SaaS infrastructure and can be reused if Google disappears (or becomes hostile) tomorrow:

- lwIP's fuzzing wrappers (lwip_fuzz etc.) live in lwip.git

- the fuzzer itself (AFL or AFL++) is free software, maintained in public git repositories, and included in distributions such as Debian

One notable exception:

- The fuzzing corpus, the collection of files generated by the fuzzer to produce different behavior in lwIP, primarily lives on Google servers, in the case of OSS-Fuzz. As far as I understand it, it can be downloaded, but this requires action on part of the lwIP maintainers. It can also be re-generated given enough CPU time.

J. Neuschäfer <jne>
Sat 08 Apr 2023 03:23:16 PM UTC, comment #5: 

Unless I'm mistaken, OSS-Fuzz is a Service as a Software Substitute.  SaaSS takes away even more computing freedom than proprietary software, we urge people to avoid such services.

Ineiev <ineiev>
Sat 08 Apr 2023 02:36:31 PM UTC, comment #4: 

Regarding the specific burdens you are concerned about:

- OSS-Fuzz has a policy that means bugs will be made public after 90 days[1], but as far as I understand it, it doesn't strictly require that bugs are fixed during this time.

- It doesn't require that all bugs are fixed, since some bugs may be considered harmless[2]. This is up to an individual project's maintainers.

- The OSS-Fuzz team might expect some basic management of bugs in the chromium.org bug tracker, i.e. marking bugs as fixed or "won't fix", etc., although OSS-Fuzz might also close bug reports automatically, when the bug is found to be fixed in the latest git version (I'm not sure about that).


[1]: https://google.github.io/oss-fuzz/getting-started/bug-disclosure-guidelines/
[2]: https://google.github.io/oss-fuzz/advanced-topics/bug-fixing-guidance#should-all-reported-issues-be-solved

J. Neuschäfer <jne>
Fri 07 Apr 2023 08:21:35 PM UTC, comment #3: 

As long as that doesn't put any burden on us (e.g. fix the reported bug in a specific timeframe, report anything back, etc.), I would be fine with that. We do have fuzzing, only we don't have the computing power to extensively run it...

Simon Goldschmidt <goldsimon>
Group administrator
Fri 07 Apr 2023 02:04:24 PM UTC, comment #2: 

The main thing in the reports is a testcase file, i.e. an input to lwip_fuzz{,2,3}. This should be enough to fully reproduce the bug.

In addition, oss-fuzz/clusterfuzz also generates a detailed report, but those seem to be limited to authorized maintainers, so I can't look at an example.


Example bug report:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57744


Unfortunately, oss-fuzz's access restrictions are implemented using Google accounts, which may be inconvenient or undesirable for some people.

J. Neuschäfer <jne>
Fri 07 Apr 2023 08:39:39 AM UTC, comment #1: 

One important detail I have experience from other projects:

It is important that the reports contain enough information about how to reproduce, and it needs to be easy to reproduce in order to debug and diagnose the problems that are detected.

Stian Sebastian Skjelstad <mywave>
Thu 06 Apr 2023 03:25:39 PM UTC, original submission:  

Google runs a service called oss-fuzz (https://github.com/google/oss-fuzz/), which offers hosted fuzzing for Free / Open Source Software projects.

To ensure that lwIP gets enough fuzzing, I'd like to try integrating lwIP into oss-fuzz.

After integration, bug report would be sent to project maintainers. Is this something you're interested in?

J. Neuschäfer <jne>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jan_breuer (Posted a comment)
  • -email is unavailable- added by ineiev (Posted a comment)
  • -email is unavailable- added by goldsimon (Posted a comment)
  • -email is unavailable- added by mywave (Posted a comment)
  • -email is unavailable- added by jne (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.12