Commit Graph

13 Commits

Author SHA1 Message Date
Akanksha Mahajan 95d582e0cc Enable io_uring in stress test (#12313)
Summary:
Enable io_uring in stress test

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12313

Test Plan: Crash test

Reviewed By: anand1976

Differential Revision: D53238319

Pulled By: akankshamahajan15

fbshipit-source-id: c0c8e6a6479f6977210370606e9d551c1299ba62
2024-01-31 12:37:42 -08:00
Peter Dillinger 61ed0de600 Add more detail to some statuses (#12307)
Summary:
and also fix comment/label on some MacOS CI jobs. Motivated by a crash test failure missing a definitive indicator of the genesis of the status:

```
file ingestion error: Operation failed. Try again.:
```

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12307

Test Plan: just cosmetic changes. These statuses should not arise frequently enough to be a performance issue (copying messages).

Reviewed By: jaykorean

Differential Revision: D53199529

Pulled By: pdillinger

fbshipit-source-id: ad83daaa5d80f75c9f81158e90fb6d9ecca33fe3
2024-01-29 16:31:09 -08:00
Adam Retter 5a26f392ca Use the correct Docker Image for RocksJava on Linux (#12169)
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/12169

Reviewed By: pdillinger

Differential Revision: D52715225

Pulled By: ajkr

fbshipit-source-id: 28476d363034fa1bb9c8c919d577c03b6391451b
2024-01-19 10:12:31 -08:00
Peter Dillinger 5a1fb5ccd6 Disable GitHub Actions jobs on forks (#12191)
Summary:
See new comment in pr-jobs.yml for context. I tried avoiding the massive copy-paste through some trial and error in https://github.com/facebook/rocksdb/issues/12156, but was unsuccessful.

Also upgrading actions/setup-python to v5 to fix a warning.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12191

Test Plan:
Here's an example of a *bad* run from my fork, prior to this change:
https://github.com/pdillinger/rocksdb/actions/runs/7303363126
Here's the "skipped" run associated with this change on my fork:
https://github.com/pdillinger/rocksdb/actions/runs/7352251207
Here's the actual run associated with this PR:
https://github.com/facebook/rocksdb/actions/runs/7352262420

Reviewed By: ajkr

Differential Revision: D52451292

Pulled By: pdillinger

fbshipit-source-id: 9e0d3db8a40e3257e6f912a5cba72de76f4827fa
2023-12-28 17:23:18 -08:00
Peter Dillinger 106058c076 Initial CircleCI -> GitHub Actions migration (#12163)
Summary:
* Largely based on https://github.com/facebook/rocksdb/issues/12085 but grouped into one large workflow because of bad GHA UI design (see comments).
* Windows job details consolidated into an action file so that those jobs can easily move between per-pr-push and nightly.
* Simplify some handling of "CIRCLECI" environment and add "GITHUB_ACTIONS" in the same places
* For jobs that we want to go in pr-jobs or nightly there are disabled "candidate" workflows with draft versions of those jobs.
* ARM jobs are disabled waiting on full GHA support.
* build-linux-java-static needed some special attention to work, due to GLIBC compatibility issues (see comments).

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12163

Test Plan:
Nightly jobs can be seen passing between these two links:
https://github.com/facebook/rocksdb/actions/runs/7266835435/job/19799390061?pr=12163
https://github.com/facebook/rocksdb/actions/runs/7269697823/job/19807724471?pr=12163

And per-PR jobs of course passing on this PR.

Reviewed By: hx235

Differential Revision: D52335810

Pulled By: pdillinger

fbshipit-source-id: bbb95196f33eabad8cddf3c6b52f4413c80e034d
2023-12-21 15:40:21 -08:00
Changyu Bi e178f9e477 Fix failed CI job "Check buck targets and code format" (#11532)
Summary:
the first CI step "Check buck targets and code format..." is failing with the following error message:
```
Download action repository 'wei/wget@v1' (SHA:c15e476d1463f4936cb54f882170d9d631f1aba5)
Error: An action could not be found at the URI 'c15e476d14'
```
Not sure why the action is lost, but it seems we can use wget directly.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11532

Test Plan: watch CI job "Check buck targets and code format"  passes

Reviewed By: ltamasi

Differential Revision: D46700626

Pulled By: cbi42

fbshipit-source-id: 53c09f27965444b533b3fe3755aec922571bba2c
2023-06-13 14:20:11 -07:00
Varun Sharma 626eaa4189 ci: add GitHub token permissions for workflow (#10549)
Summary:
This PR adds minimum token permissions for the GITHUB_TOKEN in GitHub Actions workflows using https://github.com/step-security/secure-workflows.

GitHub recommends defining minimum GITHUB_TOKEN permissions for securing GitHub Actions workflows
- https://github.blog/changelog/2021-04-20-github-actions-control-permissions-for-github_token/
- https://docs.github.com/en/actions/security-guides/automatic-token-authentication#modifying-the-permissions-for-the-github_token
- The Open Source Security Foundation (OpenSSF) [Scorecards](https://github.com/ossf/scorecard) treats not setting token permissions as a high-risk issue

This project is part of the top 100 critical projects as per OpenSSF (https://github.com/ossf/wg-securing-critical-projects), so fixing the token permissions to improve security.

Before the change:
`GITHUB_TOKEN` has `write` permissions for multiple scopes, e.g.
https://github.com/facebook/rocksdb/runs/7936368166?check_suite_focus=true#step:1:19

After the change:
`GITHUB_TOKEN` will have minimum permissions needed for the jobs.

Signed-off-by: Varun Sharma <varunsh@stepsecurity.io>

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10549

Reviewed By: ajkr

Differential Revision: D38923184

Pulled By: jay-zhuang

fbshipit-source-id: 0c48f98fe90665e53724f57a7d3b01dd80f34a93
2022-10-04 12:10:30 -07:00
Jay Zhuang e3ef41b02f Use released clang-format instead of the one from dev branch (#9646)
Summary:
We should use the released clang-format version instead of the one from
dev branch. Otherwise the format report could be inconsistent with local
development env and CI. e.g.: https://github.com/facebook/rocksdb/issues/9644

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9646

Test Plan: CI

Reviewed By: riversand963

Differential Revision: D34554065

Pulled By: jay-zhuang

fbshipit-source-id: b841bc400becb4272be18c803eb03a7a1172da6f
2022-03-01 10:51:38 -08:00
Peter Dillinger cb5b851ff8 Add (& fix) some simple source code checks (#8821)
Summary:
* Don't hardcode namespace rocksdb (use ROCKSDB_NAMESPACE)
* Don't #include <rocksdb/...> (use double quotes)
* Support putting NOCOMMIT (any case) in source code that should not be
committed/pushed in current state.

These will be run with `make check` and in GitHub actions

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8821

Test Plan: existing tests, manually try out new checks

Reviewed By: zhichao-cao

Differential Revision: D30791726

Pulled By: pdillinger

fbshipit-source-id: 399c883f312be24d9e55c58951d4013e18429d92
2021-09-07 21:19:27 -07:00
Yanqin Jin 48669be618 Update clang-format-diff.py path (#7944)
Summary:
Recent Github actions of format checking fail due to invalid location
from where clang-format-diff.py is downloaded. Update the path to point
to a stable, archived location.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7944

Test Plan: manually check the result of Github action.

Reviewed By: ltamasi

Differential Revision: D26345066

Pulled By: riversand963

fbshipit-source-id: 2b1a58c2e59c2f1eb11202d321d2ea002cb0917e
2021-02-09 12:49:38 -08:00
Xie Yanbo c9c9709a1a Update clang-format-diff.py (#7609)
Summary:
`llvm-mirror/clang` is archived. Get the `clang-format-diff.py` file from the active source.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7609

Reviewed By: ajkr

Differential Revision: D24711608

Pulled By: pdillinger

fbshipit-source-id: b115d8765ff23fbb8190290a170de21565daba84
2020-11-04 16:09:01 -08:00
Peter Dillinger c7aedf1b48 Clean up some code related to file checksums (#6861)
Summary:
* Add missing unit test for schema stability of FileChecksumGenCrc32c
  (previously was only comparing to itself)
* A lot of clarifying comments
* Add some assertions for preconditions
* Rename WritableFileWriter::CalculateFileChecksum -> UpdateFileChecksum
* Simplify FileChecksumGenCrc32c with shared functions
* Implement EndianSwapValue to replace unused EndianTransform

And incidentally since I had trouble with 'make check-format' GitHub action disagreeing with local run,
* Output full diagnostic information when 'make check-format' fails in CI
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6861

Test Plan: new unit test passes before & after other changes

Reviewed By: zhichao-cao

Differential Revision: D21667115

Pulled By: pdillinger

fbshipit-source-id: 6a99970f87605aa024fa540c78cd519ff322c3e6
2020-05-21 08:12:51 -07:00
Yanqin Jin 6acbbbf9fc Add Github Action for some basic sanity test of PR (#6761)
Summary:
Add Github Action to perform some basic sanity check for PR, inclding the
following.
1) Buck TARGETS file.
On the one hand, The TARGETS file is used for internal buck, and we do not
manually update it. On the other hand, we need to run the buckifier scripts to
update TARGETS whenever new files are added, etc. With this Github Action, we
make sure that every PR does not forget this step. The GH Action uses
a Makefile target called check-buck-targets. Users can manually run `make
check-buck-targets` on local machine.

2) Code format
We use clang-format-diff.py to format our code. The GH Action in this PR makes
sure this step is not skipped. The checking script build_tools/format-diff.sh assumes that `clang-format-diff.py` is executable.
On host running GH Action, it is difficult to download `clang-format-diff.py` and make it
executable. Therefore, we modified build_tools/format-diff.sh to handle the case in which there is a non-executable clang-format-diff.py file in the top-level rocksdb repo directory.

Test Plan (Github and devserver):
Watch for Github Action result in the `Checks` tab.
On dev server
```
make check-format
make check-buck-targets
make check
```
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6761

Test Plan: Watch for Github Action result in the `Checks` tab.

Reviewed By: pdillinger

Differential Revision: D21260209

Pulled By: riversand963

fbshipit-source-id: c646e2f37c6faf9f0614b68aa0efc818cff96787
2020-04-30 19:22:45 -07:00